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Abstract 

The paper contains a description of the maximal ideal spaces (spectra) 
AAa of bi- invariant function algebras A on a compact group G. There are 
natural compatible structures in Ma'- it is a compact topological semi- 
group with involution, polar decomposition, and analytic structure. The 
paper contains a description of A4a and related results on function alge- 
bras; for example, a bi-invariant function algebra on a connected compact 
Lie group is antisymmetric if and only if the Haar measure of its maximal 
torus is multiplicative on it. Some results are extended to the case of 
compact commutative homogeneous spaces. As a consequence, we get an 
infinite dimensional version of the Hilbert-Mumford criterion for commu- 
tative (as homogeneous spaces) orbits of compact connected Lie groups, 
where one parameter semigroup is replaced by a finite sequence of them. 
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Introduction 

Let G be a compact HausdorfF group and C(G) be the Banach algebra of all 
continuous functions on G equipped with the sup-norm. We say that A is an 
invariant algebra on G if it is a closed subalgebra of C{G) which is invariant 
with respect to all left and right translations. Invariant algebras could be defined 
as closed sub-bialgebras of C{G) with identity. We describe the maximal ideal 
spaces M.A (spectra) of these algebras. They have a natural semigroup structure 
(induced by the convolution of representing measures) and an analytic structure. 
The simplest example is the closed unit disc D = TUB, where T is the circle 
group and D is the open unit disc in the complex plane C; D is the maximal 
ideal space for the algebra of the continuous functions on T which admit analytic 
extensions to D, and a multiplicative subsemigroup of C. The polynomially 
convex hull G of any compact matrix group G is a semigroup of this type with 
respect to the matrix multiplication; for example, U(n) is the unit matrix ball 
in the space of n x n-matrices. If an invariant algebra A is finitely generated (i.e. 
generated as a Banach algebra by its finite dimensional bi-invariant subspace), 
then Ma = G for some compact matrix group G. The case of algebras which 
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are not finitely generated is more complicated, for example, it includes some 
algebras of analytic almost periodic functions on the upper halfplane in C. 

Roughly speaking, the semigroup Ma can be builded by gluing together 
complex Lie semigroups whose skeletons are compact groups. Their units are 
idempotents in A4a- There is a natural involution in A4a, which extends the 
inversion in G and fixes all idempotents. Moreover, there is a polar decomposi- 
tion: ip = gs, where Lp £ A4a, g £ G, s £ Sa, and Sa is the set of symmetric 
nonnegative elements (see ([^S]) for the precise definition). Each s € Sa is in- 
cluded to a unique one parameter symmetric semigroup, which extends to the 
right halfplane in C and induces the analytic structure in Ma- 

Let G be abelian and let Sa be the set of those one dimensional characters of 
G that are contained in A. Then, the linear span of 5*^ is dense in A. The set Sa 
is a semigroup in the dual group and Ma — Hom(5'yi, U), where D is considered 
as a multiplicative semigroup with identity and zero, and the homomorphisms 
are assumed to be nonzero. 

Let G be a Lie group and T be a maximal torus in G. Then any G-orbit 
{g~^sg : g £ G} of s £ Sa intersects the set T n Sa, where T is the ^-huU 
of T (see (IT2]) for the definition); under some additional assumptions, this is a 
maximal abelian subsemigroup of Ma- 

Invariant algebras have been studied since 50s. Initially, invariant algebras 
on abelian groups were considered as a natural generalization of the algebra 
of analytic and continuous up to the boundary functions on D. In [T], Arens 
and Singer got some results similar to classical ones, in particular, an analog 
of the Poisson integral. They found the realization of Ma as Hom(S'A,D) and 
constructed the polar decomposition in IIom(5'A,D) (which appeared earlier in 
Goldman pjj and Mackey pHI)- Paper by de Leew and Mirkil contains basic 
facts concerning invariant algebras on locally compact abelian groups (mainly, in 
the framework of abelian harmonic analysis). They also proved in [19j that any 
SO(n)-invariant algebra on a sphere S"'~^, n > 2, is self-adjoint with respect 
to the complex conjugation. Similar results were obtained by Wolf [55] and 
GangoUi [5 . Wolf characterized compact groups G having the property that 
each invariant algebra on G is self-adjoint; GangoUi independently proved that 
this property holds for connected compact semisimple Lie groups. Due to the 
Stone- Weierstrass theorem, if A is self-adjoint, then it consists of all continuous 
functions on G which are constant on cosets of some normal subgroup of G. Here 
is Wolf's condition: the image of any one dimensional character of G is finite. 
As an easy consequence of the results of this paper, we get another version of the 
criterion: the centre of G is profinite (hence, the two conditions are equivalent, 
but it is not difficult to check the equivalence directly). Further, Rider [22] 
proved that a compact group which admits an antisymmetric Dirichlet invariant 
algebra is connected and abelian (by definition, a Dirichlet function algebra 
has no nontrivial real orthogonal measure; an antisymmetric function algebra 
contains no nonconstant real functions). Antisymmetric invariant algebras were 
characterized by Rosenberg in [24] by three conditions which are too complicated 
to be formulated here; [24] also contains a generalization of [22]. It was noted in 
jllj and [7] that an invariant algebra A is antisymmetric if and only if the Haar 
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measure is multiplicative on A (see Section 4). Also, the paper [TT] contains a 
construction for analytic discs in the maximal ideal spaces of invariant algebras 
on locally compact abelian groups. 

The approach based on the observation that Ma has a natural semigroup 
structure was used in [7] where the case of noncompact groups was mainly con- 
sidered ([7] contains an error concerning the structure of Lie algebras admitting 
an invariant cone which is essential only for noncompact groups and does not 
affect the main results). The results on compact groups were announced in [5]; 
their proofs were published in [5]. This paper, in particular, contains a new 
exposition of an essential part of these results (0 is hardly accessible). 

There are several fields which are closely connected with the object of this 
paper. An evident one is the class of invariant algebras on homogeneous spaces 
(in particular, algebras of CR-functions; this involves the problem of description 
of the polynomially convex hulls for orbits of compact groups, etc.). This case 
essentially differs from the case of bi-invariant algebras (see ^7\, [1^). The 
last section contains a partial extension of the results to invariant algebras on 
commutative (spherical, multiplicity free) homogeneous spaces (the group G 
is spherical as the homogeneous space G x G/G). As a consequence, we get a 
version of the Hilbert-Mumford criterion for Banach spaces and spherical orbits. 

The paper is organized as follows. 

Section 1 contains the notation, the definitions and some known facts. 

In Section 2, the basic properties of Ma as a semigroup with involution are 
found. It can be realized as the semigroup of those endomorphisms of A which 
commute with all left translations. These automorphisms are bounded with 
respect to L^(G)-norm. Hence they extend to the closure of A in L'^{G), 
and the involution agrees with the conjugation for operators in the Hilbert 
space H^. For abelian G, there is another natural way to define the structure 
of a semigroup with an involution in A^^^; it is proved that the two structures 
coincide. Also, we show that an invariant algebra is the same as a closed sub- 
bialgebra with identity of the Hopf algebra C{G). 

The closure of the restriction of an invariant algebra to a closed subgroup is 
an invariant algebra. In Section 3, its maximal ideal space is identified with the 
yl-hull of this subgroup. The averaging by left over a closed subgroup H C G 
defines a projection to the algebra A^ of all left i/-invariant functions in A. It is 
proved that the dual map Ma —>■ Ma" is surjective. Moreover, the semigroup 
Ma acts by right on Mah in such a way that the orbit of any point in the 
Shilov boundary coincides with Ma"- The averaging over normal subgroups 
makes possible to realize the compact semigroup Ma for an invariant algebra 
A on an arbitrary compact group G as the inverse limit of maximal ideal spaces 
of invariant algebras on Lie groups. 

In Section 4, the results of the previous section are applied to the studying 
of idempotents in Ma- It is proved that A is antisymmetric if and only if the 
Haar measure on G is multiplicative on A, and this is equivalent to the condition 
that Ma is a semigroup with zero. The set of idempotents is a complete lattice 
with respect to the natural order in it. 

The polar decomposition in Ma and one dimensional analytic structure are 
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constructed in Section 5. 

Section 6 contains the main results of the paper. We assign to each idem- 
potent j several objects: a group C Ma with the identity element j, a cone 

in its Lie algebra g\ sets = cxp(iCJ) and = , etc. The cone 
is pointed, closed, convex, Ad(G-')-invariant, the set is a semigroup in the 
complexification of G^ , and P^ is naturally embedded to A^a- This defines the 
analytic structure in Ma- Any idcmpotcnt in Ma is joined with the imit of G 
by a chain of one parameter semigroups. Also, the section contains a criterion 
of the antisymmetry for the invariant algebras on connected Lie groups, which 
is stronger than the criterion of Section 4: A is antisymmetric if and only if 
the Haar measure of a maximal torus T is multiplicative on A (equivalently, if 
the restriction of ^ to T is antisymmetric). The A- hull T of T is an abelian 
subsemigroup of Ma such that each symmetric s G Ma is conjugated to some 
element of T; moreover, under some additional assumptions, T is a maximal 
abelian subsemigroup of Ma- 

In Section 7, the case of tori is considered. Then Ma = Hom(5', ID), where 
S = Ar\G and G is the dual to G group, which is embedded to C{G)- The major 
point is the identification of idempotents. Any idempotent corresponds to the 
characteristic function of a semigroup PCS such that 5 \ P is a semigroup 
ideal. They can be found by a procedure, which is described in this section. If 
A is finitely generated, then the procedure consists of only one step. 

Section 8 contains several illustrating examples and some additional results. 
In particular, it is proved that each invariant algebra on G is self-adjoint if and 
only if the centre of G is profinite; also, the section contains a description of 
antisymmetric Diriclilet invariant algebras on Lie groups. 

In Section 9, we consider invariant algebras on spherical homogeneous spaces 
of connected compact Lie groups (they are also known as commutative or mul- 
tiplicity free homogeneous spaces). This class includes the groups G considered 
as homogeneous spaces of G x G, where G acts on itself by the left and right 
translations. They has been studied last decade from various viewpoints. Some 
of the results above hold for invariant algebras on spherical homogeneous spaces 
(for instance, an invariant algebra is antisymmetric if and only if the normal- 
ized invariant measure is multiplicative; for a generic homogeneous space, this 
is not true). Using them, we get an infinite dimensional version of the Hilbert- 
Mumford criterion for the spherical orbits of compact groups. In it, one param- 
eter semigroup is replaced by a finite sequence of them. An example shows that 
the finite dimensional version does not hold for Banach spaces, in any sense. 

1 Preliminaries 

Everywhere in this paper, G denotes a compact Hausdorff topological group 
with identity element e, and G(G) is the space of all continuous functions on G 
endowed with the sup-norm, which is denoted by || || (other norms are indicated 

explicitly). For fmiction spaces L and M, LM or L • M is the linear span 
of products uv, where u G L, v £ M. Similarly, ii A,B C G, then AB = 
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A ■ B = {ab : a £ A, 6 G B}. Let G be the dual object to G, i.e., the set 
of classes of equivalent irreducible complex linear representations of G. For a 
finite dimensional representation r, Mr, Vr, { , Xrig) = TrT(g') denote the 
space of the matrix elements, the space of the representation, the invariant inner 
product in it, and the character, respectively. The following equalities hold for 
all finite dimensional representations r, a: 

MrCSa = MrM„, Xr^a = XrXcr- (1) 

If T € G and cr is a continuous representation of G (in general, infinite dimen- 
sional), then Pr = jQ<y{g)xT{g) dg, where dg denotes the Haar measure in G, 
is the projection to the r-isotypical component in V^. The convolution fj,* v of 
finite regular Borel measures /i, on G can be defined by the equality 

Ig /(f) dn * v{g) = Ja^G fidh) dn{g)dv{h), (2) 

where the space M{G) of these measures is identified with the dual to C{G) 
Banach space. Equipped with * , M{G) is a Banach algebra. The spaces C{G), 
LP{G,dg), p > 1, naturally embedded to L^{G) C M{G), are subalgebras of 
M{G) (in the sequel, we denote LP{G,dg) by Lp{G)). Spaces Mr, t G G, are 
their minimal ideals ("ideal" means "two-sided ideal"). In particular. 

By Peter- Weyl Theorem, 

i'(G)=E.eG®^x, (4) 
where the sum is orthogonal with respect to the standard inner product 

{u,v) = / u{g)v{g)dg (5) 
Jg 

and the bar denotes the complex conjugation. Throughout the paper (except 
for the last section), a function space on G is called invariant if it is bi- invariant, 
i.e. invariant with respect to left and right translations defined by 

LJ{h) = f(gh), Rgfih) = f{hg), 

respectively. Note that L is not a representation. For a function space F, set 

SpF = {reG: MrCF}, 

^fin = E.eSpF^r, (6) 

where the sum is algebraic (i.e., the set of finite sums of vectors in summands). 

Throughout the paper, "weak" refers to the *-wcak topology of the dual 
space, and "strong" to the strong operator topology in the space BL{X) of 
bounded linear operators in a Banach space X. The dual to X space is denoted 
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by X' . All representations are supposed to be strongly continuous. For a rep- 
resentation p, we denote by Sp p the set of its irreducible components (without 
multiplicities). The proof of the following proposition is omitted since these 
facts are well-known (see, for example, [IS])- 

Proposition 1. A closed subspace F of C{G), or Lp{G), 1 < p < oo, or weakly 
closed subspace of M{G), is an ideal of the convolution algebra M{G) if and 
only if it is invariant. Furthermore, the space -Ffin is dense in F for any closed 
invariant space F (in the norm topology if F C_ LP{G) or F C_ C{G) and weakly 
ifFC M{G)) and 

Ffi„ = FnC(G)fi„. (7) 

A simple computation with matrix elements shows that each space Mt, t ^ G, 
is invariant with respect to the antilincar involution 

r(5)=7(5^- (8) 
Proposition [T] and ^ imply the following corollary. 

Corollary 1. F* — F for any F as above. □ 

The space F^-a could be defined as the set of all functions in F such that the 
linear span of their left and right translates is finite-dimensional. 

The maximal ideal space M.a {spectrum) of a commutative Banach algebra A 
with identity element 1 is the set of all nonzero multiplicative linear functionals 
(i.e. Ma — Hom(A,C)) equipped with the weak topology of the dual space 
A' . For a Hausdorff compact space Q, M{Q) denotes the dual to C{Q) space 
of all finite regular complex measures on Q. If (ys e Ma, then — 1- Since 

(p(l) — 1, any norm preserving extension of (p from A to C{Q) is a positive 
measure of the total mass 1. These measures are called representing] the set 
of all representing measures for (p £ Ma is denoted by Mip. This is a weakly 
compact convex subset of M{Q). A measure that is representing for some 
if Ma will be called multiplicative. Every x G Q corresponds the evaluation 
functional 

evxif) = fix). 

The Dirac measure at x is representing for ev^- The image of Q under the 
mapping ev : x e^x can be identified with the factor of Q by the following 
equivalence: 

x^y ^ f{x) = f{y) for all f e A. 

If each class is a single point, then A is said to be separating. In this case, ev is 
an embedding and Q may be considered as a subset oi Ma- Let ^ be a closed 
separating subalgebra of G{G). The Gelfand transform A C{Ma) is defined 
by f{(p) — fif). In most cases, we omit the hat. 
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We say that A C C{G) is an invariant algebra, if it is a closed invariant 
subalgebra of C(G) that contains constant functions, li A is an invariant algebra 

A . . . 

on the group G, then the relation ~ above is invariant. Hence, the equivalence 
class of the identity e is a closed normal subgroup N and the algebra A may 
be considered as a separating invariant algebra on the group G/N . Clearly, an 
invariant subspace A C C{G) is an algebra if and only if A^jj is an algebra. It 
follows from ([1]) that Afin is an algebra if and only if Sp A contains irreducible 
components of r eg) cr for every r, cr G Sp A. 

Let 5 be a semigroup or an algebra with unit e. Then, x e 5 is said to be 
invertible, if there exist y, z € S such that xy — zx — e (then y — z). The set 
of all invertible elements in 5 is a group, which we denote by . 

Throughout the paper (except for the last section), A is an invariant algebra 
on G, and is its closure in L^{G). It follows from ([3]), (|3]), and Proposition[l] 
that 

where the sum is orthogonal. 

Let /J, € M{Q), where Q is a compact Hausdorff space, 0, $ be a continuous 
mapping from Q to a Banach space X and a strongly continuous mapping 
4> : Q ^ BL{X), respectively. Then their integrals are defined by 

^( ^ </)(g) dfi{q)) = ^(0((j)) dfi{q) for aU ^ 6 X' , (9) 

( y" ^{q)dn{q)^X = J <i>{q)xdn{q) for all x G X (10) 

The latter formula extends a strongly continuous representation t to the con- 
volution algebra M{G): 

riiA - / r{g)d^Ji{g). (11) 
Jg 

The unit circle and the open (closed) unit disc in C are denoted by T, D 
(B), respectively. In most cases, they are considered as subsemigroups of the 
multiplicative semigroup C. Also, C'^ (the closed right halfplane in C) and 

— [0, oo) are equipped with the structure of additive semigroup. 

We need some basic facts on function algebras; for more details, see Ch. 
2]. Let Q be compact and Hausdorff, A C C{Q) be a closed subalgebra. The set 
-B C Q is called a peak set for A if there exists f & A such that f{x) = 1, x € E, 
and |/(a^)| < 1, x ^ E. We say that / is a peak function for E. A peak point is 
a peak set consisting of a single point. A p-set is a set which can be realized as 
the intersection of a family of peak sets; a p-point is a point with this property. 
Clearly, p-sets are closed. If A is separating, then for each / G A the function 
I/I attains its maximal value at some p-point. In particular, the set of these 
points is not empty. The Dirac measure at a p-point x is the unique representing 
measure for the eva;. A metrizable Q contains a peak point. Modulo these facts, 
the following lemma is obvious. 
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Lemma 1. Let A be a separating G-invariant algebra on a homogeneous space 
M = G/H of a compact group G. Then each point of M is ap-point. Moreover, 
if G is a Lie group then all points of M are peak points. 

Let E Q he closed and B be the closure in C{E) of the restriction A\e- 
Then every G M.b can be considered as a functional on A. This defines the 
embedding Mb ^ Ma- Its image consists oi ip € Ma that admit continuous 
extension to B. This set is called the A-hull of E. It is denoted by i?, and can 
also be defined by 

^ = {</.eAlA: |(^(/)| <sup|/(x)| for all /£ A}. (12) 

Since A can be considered as a subalgebra of C{Ma), the definition can be 
apphed to subsets of Ma- A set C Ma is called A-convex if E = E. 
Replacing A by the algebra P{V) of all holomorphic polynomials on a complex 
linear space V, and Ma by V, we get the definition of the polynomially convex 
hull of a compact set E cV. 

Lemma 2. Let E be a closed subset ofQ. Then ip £ E if and only if there exists 
fJ. G Mip such that supp ^ Q E. If E is a p-set and Lp £ E, then supp /i C E' for 
any £ M^p. 

Proof. If ^ e Mip and supp/z C E, then the inequality \ip{f)\ < \\f\\ is obvious. 
Hence, (p £ E. If (p G E, then p2)) and the Hahn-Banach theorem imply 
the existence of a norm preserving extension /i of 93 to C{E). Since = 
93(1) — 1, this proves the first assertion. Proving the second, we may assume 
without loss of generality that £^ is a peak set. Let / be a peak function for 
E. Then ip3{f ) — 1, since /j^ is identity element of the restricted algebra. If 
X ^ E, then l/(a;)| < 1 by definition, hence, the assumption supp/i ^ ^ implies 
\jfdix\<l. □ 

We do not use any deep theorem of the theory of topological semigroups or 
measure convolution semigroups; it is sufficient to mention following elementary 
facts: 1) each compact topological semigroup contains the unique minimal ideal 
which is the union of groups; 2) each convolution-idempotent positive measure 
on G of total mass 1 is the Haar measure of some closed subgroup. Proofs can 
be found in [15] and [14], respectively. An element e of a semigroup S is called 
zero if es = se = e for all s G S. For a subset X C S let 

Zs{X) = {z e 5 : zx ^ xz for all x e X], 
Ns{X) = {z e 5 : zX = Xz}. 

If 5* = Mai then the index will be omitted. One parameter semigroup in a 
topological semigroup 5 is a continuous homomorphism R"^ — > S. 

The analytic structure in Ma is a mapping \ : D ^ Ma, where Z? is a 
domain in some complex manifold, such that f{X{z)) is analytic for all f G A. 

Lie algebras of Lie groups are realized as algebras of left invariant vector 
fields and are denoted by the corresponding lowercase Gothic letters. 
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The relative interior of a set X in a real vector space V is its interior in its 
linear span. It will be denoted by IntX. A cone C C T/ is a subsemigroup of 
the additive group of V which is invariant with respect to dilations v —>■ Xv, 
A > (hence we consider only convex cones). There is the natural preorder in 
a cone C: 

u < V ■<=^ —£u + (1 + e)v G C for some e > 0. (13) 

It defines the equivalence whose classes are called faces. For x G C, we denote 
by Fx or Fxfi the face that contains x. Clearly, each face is open in its linear 
span, the closure of in C coincides with the union of those faces of C which 
are contained in the linear span of F^ , and with the set 

Fx = {v^C : y<x}. (14) 

We shall say that F^ is a closed face and denote by the family of all closed 
faces of C . 

A closed cone C is called pointed if C fl (— C) — (in other words, if {0} is 
a face of C). An arbitrary cone is pointed, if its closure has this property. 
Condition for cones is evidently equivalent to the following one: 

u < V —ku + Iv E C for some k,l eN, (15) 

where N = {1,2,...}. The definition (fT5| . hence ([M]) . can be applied to semi- 
groups in Z". The group is always assumed to be canonically embedded to 
K". Any semigroup S* C Z" has the asymptotic cone 

oo 

a(5) =clos U i5CM". (16) 

n=l " 

Clearly, a{S) is a closed convex cone, and S C a{S). Being convex, a{S) 
coincides with its bi-dual; the dual to a set X cone X* is defined by 

X* = {ye M" : {x, y) > for aU x G X}, 

where ( , ) is the standard inner product in M". Obviously, X* = (closX)* 
and X* = {^Xy for all n G N. Therefore, 5** = a{S)\ and this gives another 
definition of a{S): 

a{S) = S*\ 

2 Semigroup structure in maximal ideal spaces 

The algebraic tensor product C{X) C{Y) (over C), where X, Y are Hausdorff 
compact spaces, can be identified with the linear span of functions f{x)g{y) on 
X X y, where / G C(X), g G C{Y). Thus the space C(X x Y) can be considered 
as a completion of C(X) ® C{Y). We shall denote it by C(X) ® C[Y) and 
identify with C(X xY). If X = y = G, where G is a compact group, then 
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the group multiplication G x G G induces the comultiplication l : C{G) — > 
C(G) § C(G) = C(G X G) by 

^if)i9,h)^f{gh), /eG(G). (17) 

The space G(G) with the pointwise multiplication of functions, this comultiph- 
cation, unit 1, counit 5e, and the antipode mapping f{g) /(.9~^) is a Hopf 
algebra. In any bialgebra X, the formula 

^I * i^if) = (m ® i^Mf)), ti,iyeX\ feX (18) 

defines a multipHcation in X'. li X — C{G), then coincides with ([2]). For 
a subspace i C G(G), let L §> L be the closure of L ^ L in C{G) §) G(G). We 
say that A C C{G) is a sub-bialgebra if 

1 e A, (19) 
A • A C A, (20) 
i(A)CA®A (21) 

Clearly, t is isometric. By (IT7|) and (fT5|) . 

11^*^11 < IIVIIII^II. (22) 

Proposition 2. ^ closed subspace A C C{G) is an invariant algebra if and 
only if it is a sub-bialgebra of C{G). Then A-^ is an ideal in M{G) and the 
algebra A' with the multiplication (jlSp is isomorphic to M{G)/A-^. 

Proof. Since and pp]) is exactly the assumption that A is an algebra with 
the unit 1, it is sufficient to prove that (PTjl holds if and only if A is invariant. It 
follows from ([2]) and (O that {u (g) v, l{w)) = {u*v, w) for all u, w, w G G(G)fin- 
By ©, Q and ((181), 

t(M^) C Mr ®Mr, red. (23) 

If A is invariant, then (|23p . taken together with Proposition[l] (particularly ([7])), 
implies (PT|) . Conversely, (pij) . (fT5|) and obvious relations 

® M(G) ±A(g) C{G), M{G) (g)A^ ± C{G) ® A 

imply that A^ is an ideal in M{G). Since A^ is weakly closed, it is invariant by 
Proposition[T] Hence, the same is true for A. It follows from ([2]), (fT8|) . ([2T|l that 
the restriction of linear functionals to A is a homomorphism M{G) A' . □ 

By Corollary [U A* = ^; evidently, * is an antilinear automorphism of A. It 
induces an involution in Ma by 

^*(/)-R7^, (24) 
where the bar denotes the complex conjugation. 



10 



Theorem 1. Let A be a separating invariant algebra on a compact group G. 
Then the space Ma with the multiplication (jlSp is a compact topological semi- 
group. The mapping ev realizes an isomorphic embedding of G to A4a- The 
identity element e £ G is the unit of Ma, and M~^ = G; moreover, conditions 
(/?,'!/'€ Ma and tp * ip ^ G imply ip,ip € G. The involution is an involu- 
tive antiautomorphism of the semigroup Ma, which coincides with the inversion 
g g^^ on G. 

Lemma 3. For each ip G A' there exists the unique continuous linear operator 
R^p : A ^ A satisfying the equality 

^{R^f) = ^ * Vif) (25) 

for all ip G A' and f A. The mapping R : ip ^ R^p is the extension of the 
right regular representation R which can also he defined by precisely, 

R^= [ Rgdt,{g) (26) 
Jg 

for any extension fi G M{G) of the functional ip from A to G{G). Furthermore, 
\\R<fi\\ ^ W^W, Rf commutes with all Lg, g d G, and the mapping R : ip —>■ R^ 
restricted to the unit ball B C A' is a homeomorphism between B with the weak 
topology and the set 

53 = {T e BL{A) : \\T\\ < 1, TLg = LgT for all g e G} 

with the strong operator topology. The inverse mapping R^^ corresponds to each 
T G 05 the functional 

R~\T): f ^Tf{e). (27) 

Proof. For any fixed p and /, the right side of ([25]) is weakly continuous on 
Ip due to (fT5|) : clearly, -0 ¥'(u ^ v) is weakly continuous on ip, hence, this 
is true for finite sums '^Uk ® W/t and their limits in the norm topology, in 
particular, for Thus, pSjl correctly defines R^f € A. Obviously, R^ 

is linear. Since A-^ is an ideal, it follows from (pS)) . (jlOp and PH)) that the 
integral in (l26l) is independent of the choice of /i, and the simple calculation 
with HI]) shows that ^ holds for the right side of The inequality 

implies \\R,p\\ < W^W- Using the approximation of as above and applying 
©, dm) and Proposition H for all g,h e G, p e A' and f e A, we get 

6g * p{f) ^Sg® Piiif)) = p{Lgf). ([23, 

R^fig) = 5g{R^f) = 5g * p{f) = p{Lgf). (28) 

Hence, R^Lhf{g) = p{LgLhf) = p{Lhgf) = R^f{hg) = LhR^f{g), and R^ 
commutes with all i/,., h G G. Pick e > and f G A. There exists a neigh- 
bourhood U B e and gi, . . . ,gn G G such that \\Lhf — f\\ < £ for all h € U 
and 

n 

G=[jUgk. (29) 

fc=i 
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Inequalities \{ip — Lp){Lg^f)\ < e, k — 1, . . . ,n, define a weak neighbourhood V 
of (/3 G Z5. If g G Ugk then g = hgk for some h E U and for each tp € V D B 

mf{g) - Kf{9)\ = u - f){L,f)\ < mug, - Lgjf)\ + 

M^{Lg,(LU-f))\ < \\Lnf - f\\+e+\\Lnf - f\\<ie. 

By (1211), this is true for all g G G, thus \\R^f - R^J\\ < 3s. Therefore, R is 
continuous. Due to the inequality \\R,p\\ < \\ip\\, R maps B to S. Since B is 
compact, i? is a homeomorphism if the inverse mapping exists; to prove this, 
note that R^{e) = by (P5)) and (pS)) . and that the norm of the functional 
/ Tf{e) is not greater than ||T||. □ 

Let dJlA be the set of all continuous nonzero endomorphisms of the algebra 
A that commute with left translations. We endow TIa with the strong operator 
topology. Clearly, 3Jlyi is a semigroup and VJIa C S. A representation r of 
the semigroup Ma in a Hilbert space H is said to be a *-representation if 

T{ip*) = t{(p)* for all (fi G Ma- 

Theorem 2. The mapping R is a topological isomorphism between A4a cind 
DJIa- For each (p G A4a, Rip admits the unique continuous extension to H^. 
This defines a * -representation R of Ma that realizes a homeomorphic and 
isomorphic embedding of Ma to BL(i?^). 

Proof. Due to Lemma [31 it is sufficient to prove that R{Ma) — to check 
the first assertion of the theorem but this is an easy consequence of ((28l) and 
([27l) . Formula ([26l) defines an extension of R to H^, which is evidently strongly 
continuous and unique. Since R is one-to-one on Ma and Ma is compact, R is 
homeomorphic. It follows from ^ and (l24l) that 6* = Sg-i for all g G G; hence, 
for if = evg we have 

R^' = r;, (30) 

where i?* is the adjoint of Rip in the Hilbert space H^. Since any representing 
measure is positive and the mappings ip ~> ip* and R,p —^ i?* are antilinear and 
continuous, ((26l) imphes ([30l) for tp £ Ma- □ 

Corollary 2. Let p be a strongly continuous representation of the group G in 
a Banach space. If Sp p C Sp^, then formula (|lip . with ^ G Mip, defines an 
extension of p to a strongly continuous representation of Ma- If P is unitary 
then the extension is a * -representation. 

Proof. The assumption Sp p C SpA implies ker p D A^ , where p in the left- 
hand side is the extension of p to M{G) defined by (fTTl) . Hence, the right-hand 
side of (|lip is independent of the choice of /i G Mip. Thus, the extension of p 
to Ma is well-defined. Standard arguments show that it is strongly continuous 
and, if p is unitary, a *-representation. □ 
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Proof of TheoremUi The multiplication in Ma is continuous due to Theorem[5] 
since the unit ball in BL(A) with the strong operator topology is a topological 
semigroup. By and ([T7]), Sg*dh — Sgh for all g,h G G; since A is separating, 
ev is one-to-one. Thus, ev is an isomorphic embedding. For each g G G, the 
Dirac measure 5g is the unique representing measure for ev^; due to Lemma [TJ 
Let ip,ip & -Ma, M £ ^ ^ -^i/>- Since /j, and v are positive, 

supp ^ * V — supp /i • supp J'. 

If (y9 * i/i e G, then supp /i * is a single point. Hence, the same is true for supp ^ 
and supp v. Thus, Lp^if) £ G. The remaining assertion was already proved in 
Theorem H □ 

For abelian G, there is another way to define the same semigroup structure in 
AAa- In this case, G is the group of characters G ^ T, which can be considered 
as a subgroup of the multiplicative group G(G)~^. Characters can be defined 
by the equality 

4x) =X®X- (31) 

The set Sp A = ^ n G is a semigroup in G (obviously, the converse is also true: 
if 5 C G is a semigroup that contains identity of G, then the closure of its 
linear span in G(G) is an invariant algebra). The restriction of G M^a to 
SpA defines a homomorphism SpA — > D that satisfies the condition (^(1) = 1. 
The set Hom(SpA,D) of all nonzero homomorphisms SpA — > D is a topological 
semigroup with respect to the pointwise multiplication and the topology of 
pointwise convergence. The essential part of the following theorem was already 
proved in |T]. 

Theorem 3. // G is abelian, then the restriction of functionals to Sp A C A 
defines an isomorphism p : A4a — *■ Hom(Spj4,D) of topological semigroups. 
For any ip G A4a and x ^ S, 

P(^*)(X)=^, (32) 
where the bar denotes the complex conjugation. 

Proof. Since S = SpA = Gr\AcA, the topology of pointwise convergence 

on S is weaker than the weak topology in A4a- Therefore, p is continuous. 

Further, p is one-to-one because Afin is dense in A. Equality ([3T|) . together 

with (|18p. implies that p is a homomorphism. Clearly, x* = X for each % S G. 

Hence, (|32|) follows from ((24|) . It remains to prove that p is surjective (recall 

that A4a is compact). Each (p G Hom(S', D) can be extended to a continuous 

linear functional on 1^{S) C 1^{G) since (p is bounded on S. Spaces 1^{G) 

and ^^(5*) are convolution Banach algebras. Evidently, (p is multiplicative on 

1^{S). The maximal ideal space of 1^{G) is known to be equal to G. By the 

1. ^ 
Gelfand-Naimark formula, hm„^oo ll/"ll;"fg) = ll/llc(G) for any / G 1^{G). 
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Thus, relations \^{f)\ - |^(r)|^ and < ||r||,i(a) imply |^(/)| < 

ll/llc(G)- Since 1^{S) can be identified with a dense subalgebra of A, ip admits 
a continuous extension to A. □ 



In the sequel, we omit * in the notation for the multiplication in A4a- The 
simplest (and leading) example, which illustrates constructions of this section, 
is the following one. 

Example 1. The algebra A(S])) of all functions that are analytic in D and 
continuous in D can be considered as a subalgebra of C(T). It is an invariant 
algebra onT, Ma{o) — ^, and the above multiplication in Ma{o) coincides with 
the multiplication of complex numbers. Indeed, for each z g D the operator Rz 
defined by Rzf{C) — f{Cz), ( G T, commutes with all translations in T and 
that RzRw = Rzw The involution * is the complex conjugation in D since it is 
the unique continuous semigroup automorphism of D which coincides with the 
inversion on T. 

Example 2. Analogously, the algebra A(Bn) of functions analytic in the open 
unit matrix ball = {Z E BL(C") : Z*Z < 1} and continuous up to the 
boundary can be considered as an invariant algebra on the group U(n), the 
multiplication (fT5|) and the involution (|24p coincides with the standard ones. 
The proof is almost word for word as above. 

We conclude this section with a description of the centre of A4a- Let Inn(G) 
denote the group of inner automorphisms x g^^xg of G; they naturally extend 
to automorphisms of A^^. 

Proposition 3. Let e Ma- Following assertions are equivalent: 



closed set is Inn(G)-invariant. Hence it contains an Inn(G)-fixed point 
which is an Inn(G)-invariant measure. The centre of the convolution algebra 
M(G) is exactly the set of such measures. Due to Proposition [21 3) implies 



3 Averaging over a subgroup and restriction to 
a subgroup 

In this section, H denotes a closed subgroup of G. The left averaging operator 
over H is defined by 





2) is trivial. If 2) is true then the convex weakly 



!)• 



□ 




(33) 



14 



where dh is the Haar measure on H. It is well-defined in any left invariant closed 
subspace of C{G). li h G H, then L^Lh = LnLh ~ Lh; moreover, = Lh, 
and Lh commutes with i?^ for all 1^9 G A^^. If i7 is normal then Lh commutes 
with left and right translations and coincides with the (naturally defined) right 
averaging operator. 

Proposition 4. The space B = LhA is a closed right invariant suhalgebra of 
A. The class 

H' = {heG: h^e} 

includes H , is a closed subgroup of G, which is normal if H is normal, and a 
p-set (a peak set, if G is a Lie group). Moreover, Lh = Lh' in A and 

B = {feA:Lhf^ffor all h e H'}. (34) 

The equality H' = H holds if and only if H is a p-set for A. 

Proof. Since Lh commutes with Rg for all g € G, the algebra B and the equiv- 
alence ~ are right invariant. Taken together with the inclusion e £ H' , this 
implies H'h = H'h~^ = H' for all h G H' . Thus, H' is a subgroup. Obviously, 
it is closed and includes H . If H is normal, then Lh commutes with Lg for all 

g G G, hence B, ^ are left invariant and H' is normal, li h <E H' and f € B, 
then Lhf{e) — /(e) by definition of H'\ replacing / by Rgf, g G G, we get 
Lhf = / and, integrating over H' , Ln'f = f ■ This proves the nontrivial part 
of ([M)) . Thus, B is the set of all functions in A that are constant on classes 
H' g for all g £ G and can be considered as a separating G-invariant function 
algebra on the homogeneous space M = H' \ G, where G acts on M by right. 
Due to Lemma [1] the class H' is a p-point for B on M (a peak point if G is a 
Lie group); since B C A, H' is a. p-set (peak set) for A on G. If is a p-set 
and g ^ H then there exists a peak set P such that H <Z P but g ^ P. Let / be 
a peak function for P. Then Lnf is a peak function for some peak set P' such 
that H C P' C P. Hence, Lnfie) = Lnfih) implies h e P'; therefore, g ^ H'. 
Thus, H ^ H' li H \s a. p-set. Remaining assertions are clear. □ 

The equivalence ^ naturally extends to Ma'- <p V'i if /(v') — /(V') for 
all / G B. Factorizing by -S, we get the image oi M.a under the mapping 
p : A' B' dual to the embedding B ^ A. Clearly, p{Ma) ^ Mb- In 
general, p{M.a) Mb but in the setting above p is surjective due to the 
following (obvious) equality: 

Lniab) = {LHa)b for all a G A, 6 G B. (35) 

Proposition 5. The above mapping p : Ma Mb is surjective. 



Proof. Let / be a proper ideal in B. 
in A then there exist functions 61 , . . 
ai&i + ■ • • + Onbn = 1. Then, by ([35l), 



If / does not generate a proper ideal 
.bn G / and ai, . . . , a,i G A such that 



1 = {LHai)bi H h {LHan)bn G / 
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contradictory to the assumption that / is proper. Hence each maximal proper 
ideal of B is contained in some maximal proper ideal of A. □ 

Let M = H \ G he & right homogeneous space of G. In the following 
theorem, the algebra = LhA is considered as a subalgebra of C(M) as well 
as a subalgebra of C{G). 

Theorem 4. The action of G on M extends to the action of A4a on Ad ah by 

f{^^) = (36) 

for all f e A^ , tp G AAa" : where f G M.A o,nd the right side defines the left 
one. For each p G AI , 

pMa = Mah. (37) 

Proof. Obviously, A^ is right invariant. Since A^ C A, each irreducible compo- 
nent of the right regular representation i? of G in A^ is contained in Sp A. By 
(PS)) . R extends to the representation of A4a in A^; clearly, this is equivalent to 
P6p and defines the action of AAa on Ma^^ that commutes with the mapping 
p above. Since gA4A = AAa for any g G G, ((37)l follows from Proposition [5l □ 

Proposition 6. If H is a closed subgroup of G, then H is a closed subsemigroup 
of Ma- Moreover, the closure of the restriction A\ ^ is an invariant algebraC on 
H , and the mapping dual to the natural homomorphism A ~^ C is a topological 
isomorphism of * -semigroups A4c and H. 

Proof. According to Proposition [21 the multiplication in A4a and in Aic is 
induced by the convolution of representing measures. If -0 G H, then, by 
LemmalU there exist measures /i G M,p and G concentrated in H. Then 

supp p, * ly C H; 

hence ipip G H by the same lemma. Since Mc is compact, remaining assertions 
follow from this fact and the natural identification Aic ^ H (which holds for 
any closed subset H C G). □ 

Corollary 3. The polynomially convex hull of a compact linear group G C 
GL(n, C) is a semigroup. 

Proof. We may assume G C U(n); then Proposition [H] can be applied to the 
algebra of Example [H □ 

By the well-known structure theorem for compact groups, each neighbour- 
hood of e contains a closed normal subgroup H such that G/H is a Lie group. 
For any pair Ha ^ Hp of normal subgroups there is the natural homomorphism 
: Ga G/3, where Ga = G/Ha and G/j = G/Hp. Let Sj = {Ha}aei, 
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where / is a set of indices, be a family of closed normal subgroups in G such 
that 



(38) 
(39) 



Then G is the inverse limit of Lie groups G/H over Sj, i.e. it is topologically 
isomorphic to the subgroup of the topological Cartesian product of these groups 
consisting of families {ga)aei such that gp — ifiapiga) if Ha C Hp. By the 
following theorem, the semigroup A4a can be realized analogously. Let £ = 
£(G) be the family of all normal subgroups H C G such that G/H is a Lie 
group and let *P = ^{G,A) be its subfamily consisting of those H which are 
peak sets for A. 

Theorem 5. Let A be a separating invariant algebra on a compact Hausdorff 
group G. For any iJ g A^ — LhA is a separating invariant algebra on 
the Lie group G/H . The projection pu ■ M.A -M-a" dual to the embedding 
A^ a is a surjective continuous homomorphism of semigroups. The family 
?P satisfies (|38p and (139]) ; as a compact topological semigroup, M.a is the inverse 
limit of M-A", where H runs over^. 

Proof. Since H is normal, its Haar measure mn belongs to the centre of the 
convolution algebra M{G). The projection Ma -Mah is a homomorphism 
because the multiplication in is induced by the convolution and the measure 
ruH is convolution idcmpotcnt. Obviously, it is continuous. By Theorem it 
is surjective. Let u be the peak function for H E Sj. Then Lhu is the peak 

a" 

function for the identity element of G/H. Hence, the equivalence ~ on G/H is 
trivial, i.e., A^ is separating on G/H. If if, P 6 *p and u, v are peak functions 
for H and P, respectively, then uv has a peak on iJ n P. This proves (|38p . The 
family of operators Lh, i? G £, is an approximate identity in C{G), hence in 
A. Therefore, if 5 G G and f(g) ^ /(e) for some / G A, then Luf lg) ^ Lnfie) 
for some H as above. Taken together with Proposition [H particularly ([M)) . this 
proves (j39p . The remaining assertion is clear. □ 

The theorem above reduces most of the problems on invariant algebras to 
the case of Lie groups. To illustrate this, we prove a lemma which will be used 
in characterization of compact groups G which have the following property: 

(SW) each invariant algebra on G is self-adjoint. 

We need the following result, which is a consequence of [31 Theorem 6.1]. 

Theorem 6. Let an invariant algebra A he separating on a compact group G 
and let Ma = G. Then A = C(G). 

For compact Lie groups, this theorem easily follows from known facts on 
polynomial approximation on polynomially convex submanifolds in C". 
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Lemma 4. The condition (SW) holds for G if and only if (SW) is true for all 
groups G / H , where H runs over £. 

Proof. Let A be an invariant algebra on G which is not self-adjoint. We may 
assume that A is separating. By Theorem [SJ Ma 7^ G\ due to Theorem O 
A^^ff ^ G/H for some G *p. This proves the nontrivial part of the lemma. 

□ 

4 Antisymmetric algebras and idempotents 

Let I A be the set of all idempotents in Ma- For j G X^i, put 

Gj = {.9 G G : gj = jg = j}, Nj = ZgU) = {g e G : gj = jg}, 

G^- = {.9 e G : gj - j}, G] = {g e G : jg = j}, 
G^ = Njj = N,/Gj, ^{ipeMA-- = = ^} = jMaj, 

and denote by rUj the normalized Haar measure of Gj . There is a natural order 
in the set of idempotents of any semigroup: j < k ii jk = kj = j. 

The following proposition will be often used in this section. Hence references 
of the type "Proposition [3 x)" will be abbreviated to "x)". 

Proposition 7. Let j ^Ia- Then 

1) r = j; 

n\ *^ r^l r^r . 

^ ) \Jr j — KjJ j — j ' 

3) Gj is a p-set for A; 

4) rUj G Mj; 

5) for any k G Ta, j < k is equivalent to Gj ^ Gk,' 

6) Nj = NciGj); in particular, j G Z{A4a) if and only if Gj is normal; 

7) (/?, ■0 G M-' and (pip = j imply ip,'ijj ^ G^ . 

Proof. Since Rj is an endomorphism of A commuting with left translations and 
= Rj, it is a projection onto a closed left invariant subalgebra B C A. The 
left orbit Oj — {gj : g G G} in Ma is the homogeneous space G/G^, A\aj 
is a separating subalgebra B of G{Oj). Due to the identity f{gj) = Rjf[g), 
which holds for all / G A and g G G, B may be identified with B. According to 
Lemma[l] each point of Oj is a p-point; hence, G^- is a p-set for A. By Lemma[2l 

supp C G'j (40) 

for any ^ G Mj. Since the set Mj is weakly compact, convex and G' -invariant 
by left this implies that Mj contains the Haar measure m^- of G' . The measure 
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m^- is inversion invariant and positive. Hence (ti^)* = to' ; this proves 1). The 
equahty gj = [jg~^)* and 1) imply G' = GJ, consequently 2). Since G^ is a 
p-set for A and to' G A^^, 2) implies 3) and 4). The inclusion Gj 3 Gk is 
equivalent to mj * ruk — to^ * nij — mj] this proves 5). If g G Nc^Gj), then 
(5g * TOj = rrij * Sg, whence gj ~ jg; conversely, if gjg^^ — j, then 

supp((5g * TJij * 5g-i) = gGjg^^ c Gj 

by (go]). Thus, g € NdGj). It remains to prove 7). If /i G M^, v G Al,/,, 
then rrij * /i * TOj G Aftp and rrij * v * mj G A^^. Hence, we may assume that 
rUj * /i * TOj = /i, TOj * * TOj = V. Then ^* v = mj by (|40p . Set P = supp /i, 
Q = supp J/. By equalities above, 

GjPGj = PGj = GjP = P, GjQGj=QGj = GjQ = Q, PQ = Gj. 

It follows that P = gGj = Gjg, Q = g^^Gj = Gjg^^ for any g G P. Clearly, 
g G NalGj). Further, TOj i5g-i = to^, whence j^g^^ = j. Since ip G jAiAj , 
this implies = gj = jg G G-'. Analogously, ip = — .19^^ G G-'. □ 

Lemma 5. A closed subsemigroup S C A^^ generated by a subset QT is a 
semigroup with zero. 

Proof. By Theorem [2l the mapping p —>■ identifies A4a with the semigroup 
dJlA Q BL(_ff^), which acts in H^. Hence, © = {Rg : s G 5} is a strongly closed 
semigroup acting in H^, and ©* = 6. By 1) and Theorem [2l if j G T, then Rj 
is an orthogonal projection. Set 

L = fl R,H^ 

and let E be the orthogonal projection onto L. Clearly, L and L-^ are left 
invariant and ©-invariant; moreover, ERg = R^E = E for all s G ©. It is 
sufficient to prove that E € 6. Otherwise, there exists a neighbourhood (in 
the strong operator topology) of E in BL{H^) which separates © and E. Then 
there exist /i , . . . , /fe G L-^ such that 

||S'//||h2>1 foraU G © and ^ = 1, . . . , fc. (41) 

For T € G, set — n Mt. These spaces are left invariant and ©-invariant, 
and pairwise orthogonal. Moreover, their algebraic sum X^tgG is dense in 
L^. Since © is strongly compact, we may assume that the functions fi, . . . , fk, 
are contained in a finite sum N of these spaces. Since L D N = {0} and N 
is finite dimensional, there exists a finite set J — {ji, . . . ,jm} C T such that 
^Djej RjH^ = {0}- Then, for each f £ N, there exists j £ J such that 
ll^j./ll < 11/11- It follows that the eigenvalues of the self-adjoint operator 

Rs = Rji Rj2 ■ ■ ■ R]^ R]„, ■ ■ ■ R]2 Rjli S ^ {jm ■ . . j2 Jl ) * {jrn • ■ ■ j2 jl ) , 

in N are strictly less than 1. Therefore, \\Rs"fi\\H'^ ^ as n ^ oo for each I, 
contradictory to ([4T|) . □ 
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An ordered set is called a complete lattice if each its bounded subset / has 
the least upper bound sup / and the greatest lower bound inf /. 

Theorem 7. The set Xa is a complete lattice; moreover, sup / and inf / are well- 
defined for each subset / C X^. In particular, 2a contains the least idempotent 
e which belongs to Z{A4a)- 

Proof. If jk = kj = j for all fc G / then jip — ipj — j for all ip in the semigroup 
generated by /. Clearly, the zero of this semigroup, which exists by Lemma[51 is 
inf/. Further, sup / = inf J, where J is the set of all upper bounds for / (note 
that J ^ % because e is the greatest element of Ia and that (pk — k(p — k for 
all fc e / and (p in the closed semigroup generated by J). By 5), ^ Gj for 
all j 2a; in particular g^^Geg ^ G^ for all g G G. Hence, is normal. By 
4) and Proposition O e e Z{Ma)- □ 

A function algebra B C C{Q) is called antisymmetric if each real valued 
function f G B is constant; a set E is called a set of antisymmetry if any 
function f G B, which is real valued on E, is constant on E. Clearly, if two 
sets of antisymmetry intersects, then their union is a set of antisymmetry, and 
any set of antisymmetry is contained in a maximal one. By the Shilov-Bishop 
decomposition theorem, a continuous function / is contained in B if and only 
if f\E £ B\e for every maximal set of antisymmetry E. Let B^ be the algebra 
of all real valued functions in B. According to the Stone- Weierstrass theorem, 
B]s_ can be identified with the algebra of all real valued continuous functions on 
Ql In general, classes of ^ are not sets of antisymmetry, but for invariant 
algebras they are. 

Theorem 8. The set Ge, where e is the least idempotent, is the maximal set 
of antisymmetry for A. It is a normal subgroup, all other maximal sets of anti- 
symmetry are cosets gG^, g G G, and Ajj consists of all real valued continuous 
functions which are constant on them. Furthermore, following assertions are 
equivalent: 

a) A is antisymmetric; 

h) Ma is a semigroup with zero; 

c) the Haar measure of G is multiplicative on A. 

Proof. The subgroup Ge is normal by Thcorcm[7]and 6). The set supp/i for any 
H G A4ip and each ip G Ma known to be the set of antisymmetry (indeed, if 
f G A is real nonconstant on supp/^, then (p ((/ — f{ip))^) > 0, but this cannot 
be true since ip {f — f{f)) = and (p is multiplicative). According to 4), Ge is 
a set of antisymmetry. By Theorem[5]and TheoremlHl A''" = C{G/Ge). Hence, 
Ge is a maximal set of antisymmetry and the same is true for its shifts gG^- It 
follows that A is antisymmetric if and only if Ge = G, i.e. if b) is true; b) implies 
c) by 4). Since the support of a multiplicative measure is a set of antisymmetry, 
c) implies a). □ 
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For abelian G, there is another simple criterion of antisymmetry, in terms 
of the semigroup S" = SpA. Precisely, A is antisymmetric if and only if 

5n(-S') = {0} (42) 

(in the additive notation). Indeed, if XiX £ A, then Kex ^ A. Conversely, A 
contains all characters in the support of the Fourier transform of any f G A^ 
but for real / the support is invariant with respect to the inversion in G; hence, 
([42]) cannot be true if / is nonconstant. 

5 Polar decomposition, one parameter semigroups, 
and analytic structure 

This section overlaps with the paper [7] . 

Lemma 6. Let B C C{Q) he a function algebra (in general, nonclosed) on 
compact Q and fi G M{Q) be a probability measure on Q. Suppose supp/i = Q 
and let E be an endomorphism of B which is bounded as a linear operator 
B ^ B with respect to the norm in L^{Q,ii). Then E is hounded in B with 
respect to the sup-norm; moreover, ||£'||bl(_b) 1- 

Proof. Suppose that f £ B satisfies inequalities ||/|| < c < 1 and \\Ef\\ > 1. 
Then the set U — {q £ Q : \f{q)\ > 1} is open and nonvoid. Hence fJ,{U) > 0. 
Therefore, 

l|i?rl|L^(Q.M) = ll(i?/)"l|L^(Q,M) > y/JW), 

WrhHQ,^) < lini = ll/ir < c" O as n ^ oo, 

contradictory to the assumption. Thus ||-E||bl(S) ^1- 

Let B C C{Q) be a subalgcbra that admits an orthogonal grading by a 
commutative semigroup A (in the multiplicative notation), i.e., 

B^Er^ABr, (43) 

BfBrC Btr for aU t,r e A, (44) 
Bt±Br in L^iQ,fi) if t ^ r, (45) 

where the sum in (|43|) is algebraic. We do not assume Bt 0. 

Lemma 7. Let B he as above, x '■ A — > D 6e a homomorphism of semigroups, 
and let a linear operator E : B B be defined by Ef — x(^)/ for f £ Bt, t £ K. 
Then E is bounded in B with respect to the sup-norm; moreover, ||-B||bl(s) 1^ 1- 

Proof. A calculation shows that E is an endomorphism of B. Since x is bounded 
and Bt are pairwise orthogonal, E is bounded in B with respect to the norm of 
L'^{Q,li), and the assertion follows from Lemma [H □ 
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Set 



Sa = {s e Ma ■■ s* = s, i?, > in H^}. (46) 

We denote by C+ the closed right halfplane in C, = [0, oo). For any ip G 
Ma, <f*<f G Sa due to Theorem [51 Let 7 : Ma be a continuous 

homomorphism (i.e., a one parameter semigroup). We say that 7 is a ray if it is 
symmetric: 7(t)* = j(t) for alH > 0. Then 7(R+) C 5a. Let be the family 
of all rays in Ma- Note that 7(0) G 2a- For k E Ia, set 

m'' ^{jem-. 7(0) = k}, (47) 
'5l = Ue«^-7(]R+)- (48) 

A continuous homomorphism 7 : C+ Ma will be called a complex ray if 
/(7(z)) is analytic in the open right half-plane for all f E A. 

Theorem 9. For any s G Sa, there exists the unique ray 7 such that 7(1) = s. 
Each ray admits the unique extension to a complex ray. For any complex ray 7, 
there exists the limit 

lim 7(z) =: 7(00) G Ia- (49) 

Rc z — >oo 

Furthermore, j{it), t G M, is one parameter group in , where j = 7(0). 

Proof. Let A be the semigroup generated by the eigenvalues of Rs- Clearly, 
A C [0, 1]. For i G A, let At be the i-eigenspace of Rs in A^n (if t is not an 
eigenvalue, then = 0). We claim that these spaces satisfies assumptions of 
Lemma[7]with A^^ as B. Since Rg is self-adjoint. At _L A^ in L'^{G) if i,r G A 
and t ^ r; hence holds. Any minimal bi-invariant subspace of is Rg- 
invariant and finite dimensional; hence Afi^ = X^jgA^*- This proves (j43p . If 
p e At, q e A,., then Rs{pq) = {RsP){Rsq) = irpq, i.e. G At^. Thus (gi]) is 
true. Put 

r e^'°s*, l>t>0, 

where logt is real. Then Xz is a bounded homomorphism A — > D for any 
z G C+, and we may apply Lemma [7] to ^fin. According to it, the multiplication 
by Xzit) on At extends to a continuous endomorphism E : A ^ A. It commutes 
with left translations because Rs has this property. Hence E = R-f(z) for some 
7(2) G Ma- Since Xz+w = XzXw, z — > Rj(z) is a homomorphism C+ OJl^ 
and 7 : C+ — * A^a is a homomorphism by Theorem [21 If / G Afin, then 
/(7(z)) is analytic inside C+ being the finite sum of the type ^Cfcc''*^. The 
mapping z — > Xz{t) is continuous in for all < G A and bounded, hence, the 
homomorphism z R-yiz) is continuous with respect to the strong operator 
topology in BL(i7^). Due to Theorem[2l z 7(2) is continuous on C+. Since 
Afin is dense in A, f{'j{z)) is analytic in the open half-plane and continuous 
in for all f G A. The uniqueness of the ray follows from the uniqueness of 
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fractional nonnegative powers of a nonnegative operator in a Hilbert space. The 
analytic extension is evidently unique. Equality (|49)) is a consequence of (|50)) : 
i?^(oo) is the projection to the closure of Ai. The last assertion follows from 
Proposition [71 7) due to equalities jj(it) = j{it)j = ^{it) and j{it)j{—it) = j, 
where teR. □ 

We shall also denote 7(2) by . Clearly, s°° £ Xa- 

Corollary 4. For any s £ Sa and z E C+, Zc{s) C Zg{s^). Moreover, if 
t > 0, then Zeis) = Zais*). 

Proof. Suppose that s is a fixed point of the automorphism ip g~^(pg. Then 
s^, Rez > 0, also is a fixed point due to the uniqueness part of Theorem [9l 
and for Rez > by the continuity. Therefore, Zg{s) C Zg{s^)- If i > 0, then 
s* G Sa and the inverse inclusion holds since s 

In the proof of the following theorem, we refer to "Proposition[7l x)" as "x)" . 

Theorem 10 (Polar decomposition). For any tf G A4a, there exist s E Sa and 
g G G such that ip = gs. In this decomposition, s is determined by ip uniquely 
and gs = hs for h E G if and only if gGk = hGk, where k — s"; in particular, 
if k = e, then the decomposition is unique. 

Proof. Let R^p = US be the polar decomposition of R^ in the Hilbert space 
H"^. Then 5*2 = = R^-R^ = R^-^ by Theorem d hence, S = Rs for 

s = ((y9*(y9)2. Since Rg is unitary for any g € G, this proves the uniqueness of 
the component s G Sa in the polar decomposition (p — gs, if it exists. 

Suppose s ^ k. Then k — k'^ ~ Pick /i G g G supp/i, and set 

K — g^^p (hence p — gn), v — 6g-i * /i G M^. Then k*k = k and e G suppi/; 
this yields, respectively, 

supp(z^* * J^) C Gfc, 
supp ly C supp ly* ■ supp v = supp(i^* * v), 

where the first inclusion holds by 3) and Lemma[2] since j/* * is a representing 
measure for k*k. Therefore, suppi^ C Gk. Due to 4), this implies nk = k. 
Further, R^ = URk for some unitary operator U; since k^ = k, we have (pk = Lp 
and p — pk — gnk = gk. Thus, the decomposition exists. By 4), we may 
assume supp/;^ = gsuppmfe = gGk. Hence, the arguments above can be applied 
to each h G gGu] consequently, p = gk = hk \i hGk = gGk. If gk = hk, then 
h-^g G Gk by 2). 

The consideration above shows that the theorem holds for s — {p*p)2 g Ta- 
In general, the operator Rj^ is well-defined on the space Agn n RkA and the 
operator E = RipRj^Rk = URk admits continuous extension from Ann to H^. 
Clearly, E is an endomorphism of Afin. By Lemma El E is bounded with respect 
to the sup-norm. Since E commutes with left translations, E = R^ for some 
Ip G Ma according to Theoremj^l It follows that = k and ^s = p>. Due to 
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the first equality, we may apply the proven assertion. Hence, there exists g € G 
such that ip = gk. Then ip — ips = gks — gs. 

Evidently, gs = hs if and only if Rg = Rh on RgH^; since RgH^ is a dense 
subspace of RkH^, this is equivalent to gk — hk. The reference to 2) concludes 
the proof. □ 

If G is abelian, then Ma — Hoin(Sp A, D) by Theorem [3l and the polar 
decomposition of x S Hom(Sp A, D) is the natural polar decomposition of func- 
tions: xix) = pix)\xix)\, where p g Hom(SpA,T). This equality uniquely 
defines p{x) if x{^) 0- general, p must be extended to SpA; the existence 
of the extension follows from Theorem [TU) 



6 Invariant algebras on Lie groups 

Everywhere in this section, G is a Lie group. We keep the notation of Section [J] 
and add new: for each j G 2a, Zj is the identity component of the centre of 
Gj ; Qj , Hj , , 3j are Lie algebras of groups Gj , Nj , G^ , Zj , respectively; tt^ is the 
canonical homomorphism Nj — > G-' = Nj/Gj. It induces projections in various 
linear spaces, which will also be denoted by tTj. Further, T denotes a maximal 
torus in G. If G is connected, then T is a maximal abelian subgroup of G. 

Each T e Sp^ can be extended to a representation of Ma by Corollary [51 
For any j 2a, this defines an extension of r to the Lie algebra by 

- t(j)t(|) = r(Or(j), 

where ^ = tt^^ G g-' , ^ G Wj. The representation of G^ and is realized in the 
space — T{j)Vr- 

Theorem 11. Let 7 G 9^ and j = 7(0). There exists the unique S^-y G g-' such 
that 

7(it) = exp(i^^) = J exp{tO = expit^j, t G R, (51) 

for any f G tt"^^-, C rij . The set 

G= = : 7 e W} (52) 

is a closed convex pointed Ad{G^) -invariant cone G^ C g^ , which can also be 
defined by 

= {e G 0^' : iT{^) < for all t G SpA}. (53) 

Mappings 7 '"^'^ ?7 7(1) ^""6 one-to-one on C'-' ; they identify with 

and 5*^, respectively, where D^-* , S"^ are defined by (|47p . (j48p . 



^(0 = I7^(exp(t0j) 
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Proof. The first assertion is a consequence of Theoreni[S] (note that the mapping 
g — > jg on Nj is the homomorphism tTj). Clearly, (j53p defines a convex, closed 
and Ad(G-' )-invariant cone. If ^ £ , t € Sp A, then exp(izT(^)) is bounded in 
Vr uniformly on z G C+. Therefore, the operator group i?exp(t5)ji t G K, admits 
analytic extension to iC"*" in RjMr for all r G SpA, hence to RjAan, and this 
extension is bounded with respect to i^-norm. Furthermore, the extension is 
an endomorphism of the algebra Aj = RjAfi^ since Rexp{z£,)j € Hom(Aj, Aj) for 
2 G K and the analytic extension agrees with the multiplication of functions. 
This defines i?exp(it{)j for < > 0. It is a semigroup of endomorphisms of Agnj 
which commutes with left translations and is symmetric in all Mr, t G SpA. 
Hence, it defines a ray 7 G W (we shall write j{t) = exp{it£,)). Conversely, 
since T{'j{t)) is bounded and symmetric semigroup in Vr for each ray 7 G 
and any r G SpA, we have 

lr(7(t)) = zT(e^) < 0. 
at t=Q 

Thus, dsn) and define the same cone. If ^ G and G , then 
/(exp(z^)j) is bounded in C for each / G Afin- Since this function is continuous 
on C and analytic on C \ R, it is entire. By Liouville Theorem, /(exp(z^)j) = 
/(j), but this implies exp{z£,)j = j since Agn separates points of Ma', hence, 
^ = 0. This proves that is pointed. The remaining assertion is clear. □ 

We say that a ray 7 is j-central if 7(0) = j and 7(t) G Z{G^) for all i > 
(omitting j in the notation if j — e). 

Corollary 5. If ^ 0, then the centre of is not discrete, and there exists 
a nontrivial j-central ray 7. 

Proof. By TheoremEl G^ ^ {0}. If C £ G^ \ {0}, then 

C = / kA{h)£,dmj{h) ^ 0; 



moreover, C is Ad(G-')-fixed. Hence, exp(tC) lies in the centre of G^ and the 
semigroup Rcxp{itc) commutes with Rh for all G G^ . □ 

By Gc we denote the complexification of the group G. As a set, it can be 
identified with Gexp(ig) (we may assume that G is a matrix group). Each finite 
dimensional representation of G uniquely extends to Gc. For j G Xa let V be 
the real linear span of G^ in g-' , 

P^ = G^exp(iG^). 

Since G^ is Ad(G-' )-invariant, V is an ideal in and 

G^ exp(iG^ ) = exp(iG^' )G^ . (54) 

Let , be connected subgroups of G^ , Gj, that correspond to P , [j,, respec- 
tively. In the theorem below, they are equipped with the underlying topology 
of a Lie group. 
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Theorem 12 (Analytic structure). For each j E Ta, the set is a subsemi- 
group of Gj^, which consists of g G Gj^ such that 

M9)\\BLiv^) < 1 (55) 

for all T G Sp A. The interior of L''^ D in Lj. is an open semigroup in 
Lj,, and the natural embedding —>■ Ma defines the analytic structure, which 
is nontrivial if ^ 0. 

Proof Let g = /lexp(i^) e G^, where /i G G^ , ^ £ g^. Then r(exp(i^)) = 
T{g)T{h)~^ . Hence, T{g) E t{Ma) is equivalent to r(exp(i^)) e t{Ma)- Since 
t(/i) is isometric in , (1551) holds for exp(i^) and for g simultaneously. Further, 
(|55p is true for exp(i^) if and only if 

||T(exp(ztO)||BL(K^) <1 foralH>0 

(the norm of a symmetric nonnegative operator is equal to its greatest eigen- 
value), and this is equivalent to the inequality in ([55)) . By Theorem [TT1 P^ is 
distinguished by inequalities (|55p : hence, P^ is a semigroup in G^. Evidently, 
P^ n Lj, = cxp(iC^), and this set has nonempty interior in Lj,, which is 
also a semigroup. The embedding — > A4a defines the analytic structure in 
Ma because the mapping g —^ T{g) is holomorphic on L^, for all r G Sp A, and 
the uniform closure keeps this property. If G^ 0, then 7^ 0. Hence, is 
not a single point. □ 

Lemma 8. There exists an open subsemigroup U of Ma such that U ^ G and 
{(p*ip)'^ ~ e for all if E U . 

Proof. Since is dense in A and A is separating, there exists a finite dimen- 
sional representation r of G such that Spr C SpA and r is faithful on G. By 
Corollary [21 r can be extended to Ma- Then T{j) ^ 1 for each j ^ e m. Ia 
(this is a consequence of ([TT|). Proposition [71 4), and the choice of r). Hence, 
detT(j') = 0, and U = {ip E Ma '■ detT(iy9) ^ 0} satisfies the lemma. Indeed, 
(p{(p*ip)'^ = If for all f E Ma', since r((^) is invertible for ip E U, the idempotent 
{(p*(p)^ is also invertible. This implies {(p*(p)^ = e. □ 

Proposition 8. If A ^ G(G) then + 0. 

Proof. Due to Theorem [SI the least idempotent e does not coincide with e. 
By Proposition [HI it is sufficient to prove the assertion for G^. According to 
Theorem [8l we may assume that A is antisymmetric on G. Since each function 
which is constant on G is constant on Ma, this means that A is antisymmetric 
on Ma- Then Ma is connected since A contains the characteristic function of 
any closed and open subset of by Shilov's Idempotent Theorem. It follows 
that U n {Ma \ G) 7^ for any neighbourhood J7 of e in Ma (otherwise, G 
is open in Ma due to the homogeneity). Further, if 1^ ^ G then ip*ip ^ G hy 
Theorem [TJ Thus, Lemma [H) and Theorem [Hi imply the existence of a nontrivial 
ray 7 starting at e, and Theorem [TT] yields 0. □ 
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Corollary 6. If A ^ C{G), then there exists C, G IntC"^ such that the corre- 
sponding one parameter group in G is a subgroup of Z{G) and exp(it(), t >0, 
is a nontrivial ray in Z^Ma)- 

Proof. Combine Proposition [51 Corollary [5] and Proposition [31 □ 

Lemma 9. If j, k G Ia, j 1^ k and j ^ k, then dimCj > dimGfc. 

Proof. If dimGj — dimG^, then the homogeneous space Gj/Gk is finite and 

the algebra RkA is separating on it by Proposition [Hand Proposition [71 3). 

Each separating algebra on a finite set is evidently the algebra of all functions 
on it. Hence, the measure mj cannot be multiplicative on A, contradictory to 
Proposition [71 4). □ 

Lemma 10. Let j G Xa- If j 7^ e, then Z{G^) contains a nontrivial ray starting 
at j . Moreover, if j G Z(A4a) and j ^ e, then there exists a nontrivial ray lying 
in Z{A4a) and .starting at j . 

Proof. If dimG = 0, then G is finite. Since A is separating, A = G{G); hence, 
M.A — G and e — e. This provides the base for the induction by dimG. 
Let dimG > 0, e ^ e, and 7 be a nontrivial central ray in Ma starting at 
e (existing by Corollary [6]) . Then k — 7(00) 7^ e. If fc ^ j, then ^(t)j is a 
ray in Z{G^); moreover, it is contained in Z^Ma) if j G Z{Ma)- This ray is 
nontrivial because kj 7^ j. Let k > j. Since 7 is central, k G Z{AiA)- Further, 
dimGfc > according to Lemma [H Therefore, dim G/Gfc < dimG and the 
induction hypothesis can be applied to the algebra RkA on G'^ = G/G^, whose 
maximal ideal space can be identified with the semigroup kAiA by Theorem 21 
It follows that either j ~ e or W contains a nontrivial ray 7 in Z{G^) D kM.A- 
If j G Z{M.a), then 7 can be assumed to be central in kM.A by Corollary [SI 
then it is evidently central h\ M.a- CH 

Corollary 7. G^ ~ if and only if j — e. □ 

We say that a finite sequence 71 , . . . , 7„ is a chain of rays of the length n if 
all rays are nontrivial and 

7z(oo) = 7/+i(0), / = l,...,n-l. 

Also, we say that the chain joins 71 (0) and 7„(oo). 

Theorem 13. For any pair j, k G Ia such that j < k, there exists a chain of 
rays which joins them. Its length does not exceed dim Gj — dim Gfc . 

Proof. It is sufficient to prove the proposition for the restriction of A to Gj. 
Hence, we may assume j = e but in this case the assertion is almost evident: by 
Lemma [TOl there exists a chain of rays starting at k, and it cannot be continued 
only if its endpoint is the least idempotent e. The upper bound of the length 
follows from Lemma [H □ 
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Let (Tj denote the Haar measure of Zj. 



Proposition 9. For any j £ Za. o-j £ -Mj- 

Proof. It is sufficient to prove that Ce £ Ai^, assuming G = (then we get 
the proposition applying this to ). We use the induction on dimG. The 
assertion is evident if dimG = 0. If dimG > 0, then there exists a nontrivial 
central ray 7 G fH*^. For all f e A and t > 
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due to Theorem llll Hence, M'y{t) contains a measure concentrated on a central 
one-parameter subgroup of G. Therefore, k = 7(00) has a representing measure 
with the support in its closure that is a connected central subgroup of G. As in 
Theorem [11 k e Z{Ma), Mr^a = kMA, G^ = G/Gk and dimG'^' < dimG. 
By the induction hypothesis, the Haar measure of the identity component 
of ZQk{G^) is representing for e on G*^. We claim that Z^^ = kZ^. Indeed, Z^ = 
expQe) is the exponent of the set of Ad(G*'')-fixed points in g'^; Tr^-preimages 
in g of all these points are Ad(G)-invariant, hence contain Ad(G)-fixed points. 
Therefore, = TTfeCTj. This implies ^ Ai^. □ 

Proposition 10. For any s G Sa, there exists a connected abelian subgroup 
H (~ G such that 

seH C Z{s). 

Proof. Let j = s°, be the Poisson measure -^jr^ on R. By Proposition [9l 
Theorem |9] and ([5T|) . for some f G rij 



/ / fijexpitOdi^it)^ 

f{hexp{tO)dv{t) daj{h). 



Therefore, s has a representing measure concentrated on the closure iJ of the set 
Zj exp(R^). Clearly, Z^ is Inn(A'j)-invariant; hence, the inclusion ^ G rij implies 
that 7J is a group. Furthermore, Fl is abelian since it is compact and solvable. 
Evidently, H is connected. If /i G Zj C Gj, then hs = sh = s due to equalities 
s = js = sj and jh = hj ~ j; if h ~ exp(t^), t G K, then hs ~ ~ sh. Thus, 
FCZ(s). □ 

Proposition [10] enables to prove a stronger version of Theorem [8] for con- 
nected Lie groups. 

Theorem 14. Let G be connected, T be a maximal torus in G, t be the Haar 
measure of T , and A be an invariant algebra on G. Then the following asser- 
tions are equivalent: 
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a) A is antisymmetric; 



b) the closure of A\rp in C{T) is antisymmetric; 

c) T is multiplicative on A. 

In this case r G M.f_, where e is the least idempotent. 

Proof. Let A be antisymmetric. Then G — G^, and is the identity component 
of the centre of G. Hence, C T. By Proposition [9l G AA^. Due to 
Theorem m e is zero of A^^i; in particular, (5t *(Te £ Ale for alH G T. The convex 
weakly closed span of these measures contains r. This proves the implication 
a) c) and the last assertion of the theorem. If A is not antisymmetric, then 
it contains a nonconstant real function /. Let f{g) ^ /(h), where g,h G G. 
Since G is connected, there exist x,y € G such that xgy,xhy g T. Then 
L~^R~^f e A is a real function, which is not constant on T. Thus, b) implies 
a). It remains to note that b) and c) are equivalent by Theorem [5] □ 

For Lie groups, the ordered set Xa has some special properties. 

Proposition 11. For any k £ X^, there exists its neighbourhood U in A4a such 
that j G Z4 n Z{k) n U implies k < j . 

Proof. If jk = kj, then jk G Xa- Let p be a finite dimensional representation of 
G such that Sp p C Sp ^ and p is faithful on G*^ . Set 

W — {tp £ Ma ■ rank p{ip) > rank p{k)} . 

Then the interior of W contains k. For any nontrivial ray 7 G IH*^, the one- 
parameter semigroup p(7(i)) is nontrivial due to Theorem llll Hence, 7(00)) ^ 
W. Taken together with Theorem [131 this implies that W does not contain 
j & Xa j ^ k and j ^ k. Let j G Xa n Z{k). Then jk G Xa and jk < k. 
Therefore, jk 7^ k implies jk ^ W. Thus, any neighbourhood U CW oi k such 
that CW satisfies the proposition. □ 

Proposition 12. Let j G Xa, and let T be a maximal torus in G. Each of the 
following conditions is equivalent to the inclusion j G T: 

a) J G ZiT); 

b) T C Nj; 

c) T n Gj is a maximal torus in Gj ; 

d) Zj C T. 

In this case, Tj is a maximal torus in G^ and 

GjnZ(T) CG^. (56) 
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Proof. According to Proposition [SI T is abelian. Hence, j G T implies a). By 
a), T C Z{j) = Nj, i.e., b) holds. If T C Nj, then H = TGj is a subgroup of 
G, which is locally a direct sum of a torus T' C T and Gj. Then T and T/T' 
are maximal tori in H and H/T', respectively. Since H/T' is locally isomorphic 
to Gj and all maximal tori are conjugated, the torus T H Gj is maximal in Gj. 
Thus, c) follows from b). The implication c) =^ d) is trivial. If d) is true, then 
j G T by Proposition ^ and Lemma [H 

Clearly, T is a maximal torus in Nj. Thus, b) implies that Tj = T/ {TOGj) 
is a maximal torus in G^ = Nj/Gj. 

MteT,geG and gj e Z{T), then tgj - gjt. By d), 

/(J5J) = J fitgj) da,{t) - j figjt) da,{t) = /(^j'), f^A 

where aj is the measure of Proposition [5] This implies jgj = gj and g~^jgj = 
j. Applying Lemma [Ml and Proposition [3 3) and 4), we get g^^GjgGj = 
supp((5g-i * rrij * Sg * nij) ~ Gj. Therefore, g^^Gjg C Gj. i.e. g G Nj and 
gj e G^. This proves □ 

Proposition 13. Let T be a maximal torus in G. Then 

Sa n Z(T) = 5a n f . (57) 

Proof. Let s e 5AnZ(r). Then G Z(r) for all z e C+ due to Corollary[l In 
particular, j = s° G Z(T). By Proposition [H a), j G T. Thus, one parameter 
group j{t) = s", i G R, hes in nZ(T) C Z(Tj). Since Tj is a maximal torus 

in G' by Proposition [H 7(IR) Q Tj. It follows from Theorem [11] that s G 7(M); 
hence s G Tj C T. This proves the nontrivial inclusion Sa n Z(r) C 5a nf . □ 

The following theorem reduces many problems concerning invariant algebras 
to the abelian case. We shall say that the semigroup T below is a Cartan 
suhsemigroup of M.a- 

Theorem 15. Let T be a maximal torus in G. 

a) For each s G Sa, there exists g £ G such that g^^sg G T. 

b) Let ip G Z[T) and let ip — gs, where s G Sa, g & G, be its polar decompo- 
sition. Then s G T and gj G G^ fl Z(T), where j = s'^ G T. 

c) If G^ is connected for every j G Ta, then T is a maximal abelian subsemi- 
group of A4a. 

Proof. Since any connected abelian subgroup of G is conjugated with a subgroup 
of T by an inner automorphism, a) follows from Proposition [TOj and Lemma [5J 
If <y9 G Z{f), then If* G Z{f*) = Z{f), hence, tp*ip G Z{f). By Corollary[l 
{ip*(pY G Z{T) for all t > 0. In particular, s = {(p*(p)^ and j = s'^ belong 
to Z{T). Applying ([57]) . we get s,j G T. Since Rs and i?j are sclf-adjoint in 
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, equalities sj ^ js = s imply keri?^ = keri?^. Thus, the operator ^ is 
well-defined on RjAnn- Therefore, for all / S RjAnn and t £ T 

RtRgjf — RtRipRg = RipRs ^Rtf = RgjRtf- 

Hence, gj G Z{T) — Z{T), and a reference to concludes the proof of b). 

If is connected, then Tj is a maximal abelian subgroup of being its 
maximal torus. Therefore, Z{T) D G^ = Tj C T. Thus, b) and the assumption 
of c) imply Z(f ) = f . □ 

Of course, T need not be a maximal abelian semigroup in Ma in general. 
For instance, this is true if G is abelian disconnected. An illuminating example 
is the group G = T x Z2 and the invariant algebra A corresponding to the 
semigroup 

S = {{n, 1) : n e Z, n > 0} U {(n, -1) : n £ Z, n > 0}. 

in the group G = Z x Z2, where Z2 is realized as the multiplicative group {±1}- 
Also, A can be described as the algebra of pairs of functions /i, /2 G A{B>) (see 
Example [J) such that /i(0) = /2(0); Ma is the union of two discs, which have 
a single common point. 



7 Invariant algebras on tori 

Theorem El identifies Ma and Hom(SpA,D). If G = T", then G = Z", Sp^ is 
a semigroup in Z", and the algebraic structure of IIom(Spyl,D) is determined 
by the geometry of SpA. Main ingredients are the asymptotic cone a(SpA) 
(see (fT6)) ) and the face structure. Everywhere in this section, SpA is denoted 
by S; it can be an arbitrary semigroup in Z". We start with a description of 
subsemigroups of S corresponding to idempotents in IIom(S', D). They will be 
called faces of S. We shall prove that one can get the collection 3s of all faces 
by a procedure of step-by-step expansion. It starts with 3s = {S} and finishes 
when 3s does not change. On each step, the following operation is applied to 
the faces P that are already found: 

if e Sa{P) and P n F 7^ then add P n F to 3s. (58) 

Obviously, the procedure is finite. Since Ma is abelian, the set Sa is a semi- 
group. According to ([5^ . it can be identified with IIom(5', I), where I = [0, 1] 
is considered as a multiplicative semigroup. Each idempotent in IIom(S', I) cor- 
responds to the characteristic function kx of some set X C S: 

We denote ks by 1. 
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Lemma 11. Let 7 be one parameter semigroup in Hom(S', I). Suppose that 
7(0) = 1. Then -/{t){x) = e"*^(^), where X G S* , t > 0, x e S . Furthermore, 
7(00) = Kp, where P — S H F for some F G da(s)- 

Proof. Without loss of generality we may assume that S linearly generates R" . 
Let X €1 S. The assumption 7(0)(a::) = 1, taken together with the continuity and 
the semigroup property, implies j(t){x) > for all t > 0. Hence, a real valued 
function 

At(a;) = -log7(t)(x) 

is well defined. Clearly, At is additive on S. Thus, it extends to the additive 
functional on the cocompact discrete group S ~ S, hence to the linear functional 
on R" . Further, Xt — tXi since Aq (x) = and At {x) is an additive and continuous 
function on t for any x ^ S. Obviously, At G 5** = a(S)*. Therefore, ^{t) = 
e~*^, where A = Ai e 5**. It follows that 7(oo)(a;) = if X{x) > 0, and 
7(oo)(a;) = 1 if A(a;) — 0; in other words, 7(00) — Kp, where 

P = SnF, F ^a{S)nX~^{0). 
It remains to note that F e ^a{S) since A G S**. □ 
Corollary 8. The set 5^ is open in Sa; moreover, for any x € Inta(iS') 

51-{xGHom(5,I) : ^ 0}. 

Proof. Let x G IIom(S', I) = Sa- By definition, if x ^ 5^, then XKp = x for 
some P ^ S. By Lemma [TT] and Theorem [T51 this implies % = on IntQ;(S') n 
S. ' □ 

Proposition 14. For a set P C S", the inclusion P G J5 is equivalent to each 
of following conditions: 

1) P is a semigroup and S \ P is an ideal of the semigroup S; 

2) Kp G Hom(S',D). 

Moreover, if P CzJs o-nd P ^ S, then a{P) is a closed cone containing in some 
face F G ^a(S), F oi{S). 

Proof. Let P €z Js- Proving 1), we may assume without loss of generality that 
P is added to 3s on the first step, i.e., that P = F n 5* for some closed face 
F of the cone a{S). Then P is a semigroup and \ P is an ideal since F has 
these properties with respect to a{S). The equivalence of 1) and 2) is obvious. 
Taken together with Proposition [Til and Lemma [TTl 2) implies P G 3s- The 
last assertion follows from ([55)1 . □ 

The semigroup S uniquely determines A4a but the converse is not true (for 
example, the semigroup in Z which is generated by numbers 0, 2, 3 corresponds 
to the same maximal ideal space as the semigroup of all nonnegative integers). 
Proposition [Ml and Lemma ITTl show how to reconstruct Ma by S. We outline 
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this procedure (without proofs) and construct a kind of a hull for S which 
corresponds to the same maximal ideal space as S. Besides, we give some 
examples to illustrate the complexity of the object. 

Topologically, Ma^sGxSa factorized by some equivalence. By TheoremfTUl 
to decide if (g, s) and (/i, s) represent the same point in M.a or not one have to 
know if g~^h G , where j = s°; the component s must coincide. According to 
Theorem [11] Sa is the union of cones , j ^ Ia, which are compactified and 
glued together. Thus, to build A4a, one needs 

Ia, €a = {C^ ■■ 3 e Ja}, &a = {G' ■■ J e Ia}, (60) 

and has to know points which must be identified. The geometric object which 
contains information on Sa (precisely, Xa, €a, and the topology in the union of 
, j e I_a) is the family of asymptotic cones to faces of S: 

€s = {a{P) : PeJs}. 

Indeed, faces are in one-to-one correspondence with idempotents in A4a by 
Proposition [T31 2) and Theorem [31 For each j ^ 2a, the cone C of 
Theorem [TT] is dual to Cj € £5. The topology in the union of these cones 
relates to the topology of the pointwise convergence in Hom(S', I). It is sufficient 
to describe the closure of C"^ = C*, where Ge = a{S) (the operation must be 
repeated for all j (Hi a)- The first step of the procedure ([58)) defines idempotents 
lying in the closure of S\ in Ma- The set S\, by Theorem [TTl may be identified 
with and, by Lemma [TTl with the set 

{e^^ : X£G^}Q Hom(5,I). 

Let j G Ta belong to the closure of S\ and P be the corresponding face of S. 
Clearly, G^ contains all functionals in G* restricted to Gj but it can be wider if 

Cj = a{P) 

is a proper subset of the closed face of that includes P. The restriction of 
G* can be identified with the projection Hj{G'^)\ the closure of C"^ is the union 
of these sets. Continuing this, we get Sa- 

To find 25^, it is sufficient to know for each j E I a the annihilator Tj of G^ 
consisting of y £ G such that y{g) — 1 for all g E Gj. Then 

G^ = {5eG: y(5) = l for ally eP,} (61) 

due to Pontrjagin duality. Clearly, Tj may be replaced by Gj fl in ([61]) and 
Tj is the group generated by Gj n S. 

Let ©5 be the collection of groups Tj, j £ 3s- Families 

:Js, <ts, ©s (62) 

correspond to Ja,^Ai 25a, respectively, and uniquely determines them. Objects 
([62l) satisfy following conditions: 
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A) if j < fc then Cj is a closed cone which is contained in some face of Ck ; 

B) Cj =a(TjnCj); 

C) if j < /c then Tj C T^, 
where /c G == 3s- Set 

^ = U-,T ^he-^e i5 =r,nintQ. (63) 

Then S ^ S and S = S. The set 5 is a semigroup, and famihes ((62)) for S* 
are the same as for S. Thus, — Ma, where A is the invariant algebra 

corresponding to S. Combined with the procedure ([55]) . conditions A)-C) give 
an universal method to construct maximal ideal spaces of invariant algebras on 
tori. 

The case of finitely generated semigroups is simpler than the general one. 
We keep the notation of Theorem[I3 "compactification" means "homeomorphic 
embedding to a compact space as a dense open subset" . 

Proposition 15. Suppose that the semigroup S Qli^ is generated by its finite 
subset. Then 

a) the correspondence between Js o,'<^d'Sa(s) defined by P — Fr\a{S), where 
P G3s o^nd F e da{s); is one-to-one; 

b) 5^ is dense and open in Sa; 

c) if A is antisymmetric and separating, then G ~ L'^ , and A4a is a com- 
pactification of the domain D'^ in the complex torus Gc ■ 

Proof. Set C — a{S) and let X — {xi, . . . ,Xm} be the finite set generating S. 
Then 

S' = NxiH hNx„i, C = R+a;iH hM+a;™- (64) 

Each closed face F G of the convex closed cone G has the form F = \^ CiC 
for some X e C* (precisely, for every A e Int(i^^ n C*)). Set Xx = r\ X. 
Then 

F = ^ R+x (65) 

since A(a;) > for x <^ X\. This implies that F a(F) for P = F n and that 

^F^^c- (66) 

Therefore, the mapping P ci{P) is inverse to F — > F n 5* and a) is true. It 
follows from ([65]) that the projection C* — > F*, dual to the embedding F ^ G, 
is surjective. Combining this with Lemma [Til we get that each one parame- 
ter semigroup 7 in Hom(P, I) such that 7(0) = Kp can be realized in the form 
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Kp"f{t), where 7 is one parameter semigroup in Hom(S', I). Thus, (j66p . Theo- 
rem |9l Theorem [T3l and CoroUary [8] yields b). Due to Theorem [TOl and b), the 
set P*^ = is dense and open in Ma (in fact, this is the set U of LemmaH]). 
If A is antisymmetric, then C is pointed according to ([^^ and (IM)) . Hence, the 
dual cone C* = C"^ C g has nonempty interior in g; consequently I = g. Since 
G is connected and A is separating, G — L'^ and D"^ is the interior of in the 
group Gc- The polar decomposition on the set is one-to-one, hence locally 
homeomorphic. Therefore, D'^ is dense in P'^; this proves c). □ 

Corollary 9. IJ S is finitely generated, then the set I4 is finite. 

8 Examples 

We shall say that an invariant algebra A is finitely generated if some its finite 
dimensional invariant subspace generates it as a Banach algebra. If G is abelian, 
then this is equivalent to the condition that the semigroup S* = Sp A is finitely 
generated. It is not difficult to prove that A is finitely generated if and only if 
there exist a finite dimensional Hilbert space V and an isomorphic embedding 
of G to V{V) such that Afin = V{U{V))\^; then A4a coincides with the poly- 
nomially convex hull G of G in BL(y). Further, if A is finitely generated, then 
the closure of the restriction to the maximal torus T is a finitely generated 
invariant algebra on T (probably, the converse is also true). If G is abelian, 
then the compactification of D'^ in Proposition [13] can be identified with its clo- 
sure in the affine variety IIom(5, C), which admits a realization as an algebraic 
submanifold of C". Precisely, 

closD^ = {X e Hom(5,C) : |x(a;)| < 1 for aU x e X}, 

where X is a finite generating set. The example below demonstrates some 
typical effects that occur in the realization above. 

Example 3. Conditions p,q > 0, where p is even if q = 0, distinguish a semi- 
group S C Z^. It is generated by xi — (2,0), X2 — (0,1), X3 — (1,1), which 
satisfy the relation xi + 2x2 = 20:3. The algebra A is isomorphic to the closed 
(in the sup- norm on T^) linear span of all holomorphic monomials z^Z2 in 
except odd powers of zi. There are 4 idempotents in 2a corresponding to 4 
faces of S (two of them are trivial). Clearly, 3s = ^'^^ '^S = '^s ^'^^ 
semigroup S = {{p,q) G : p, g > 0} but the group corresponding to the 
face p > 0, g = 0, is disconnected; A4a can be realized as the bidisc |zi|, |z2| < 1 
in with identified points (±zi,0). Putting Zk = xi^k), k = 1,2,3, where 
X G Hom(5, B), we get an embedding of A4a to the variety — z| in C^; 
the image coincides with that of the mapping ^ C'^, (Ci, C2) ^ (Ci 7 C2, C1C2), 
and is distinguished by inequalities |zi|, |z2|, l^al < 1- CH 

Following three examples illuminate some properties of infinitely generated al- 
gebras, which finitely generated ones do not possess. According to the first of 
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them, S\ need not be dense in Sa ■ This is a modification of an example studied 
in [21 Section 5] with many details. 

Example 4. Let S be the semigroup of (p, q, r) G 1? such that either r > 
or r = and p, g > 0. Then £5 consists of 5 cones: r > 0; r = 0, p, g > 0; 
r = p = 0, (7>0;r = (7 = 0, p>0;r = p = g = 0. The set Sa is 
homeomorphic to the union of the unit disc D and the interval [1,2] which 
represents S'^ (with 2 corresponding to e). Hence S\ is not dense in Sa (cf. 
Proposition[T5l b)). The semigroup S is not finitely generated but each character 
X £ Hom(S', D) is uniquely determined by its values at points x\ = (1,0,0), 
xi = (0,1,0), 0:3 = (0,0,1). Indeed, for every x G S and all sufficiently large 
n,m vector x + nxi + 171x2 belongs to the semigroup generated by xi,X2,X3. 
This uniquely defines x(x) if xi^k) 7^ 0, fc = 1,2. Set Zk — xi^k), k = 1,2,3. 
If 2fc = for fc = 1 or fc = 2, then xi^ + ^k) = for any x E S; hence, 
xip, g, r) = for all p,q G Zi and r > 1 but zi, Z2 uniquely determine x(p, q, 0). 
Further, if Z3 ^ 0, then \zi\ = \z2\ — 1: otherwise, x cannot be bounded on 
ZX1+ZX2 + X3. If Z3 = 0, then there is no restriction on zi, Z2 except zi,Z2 € D. 
Thus, Ma = Ml U M2, where 

Ml = {(zi,Z2,Z3) : ksl < 1, = \z2\ = 1}, 
M2 = {(21,22,23) : ^3 = 0, \zi\ < 1}. 

Both sets have real dimension 4. Functions of A are analytic on Z3 in Mi and 
on zi, Z2 in -^2! ^ is antisymmetric. □ 

If S is not finitely generated, then Corollary [5] also need not be true. Note that 
Xa is at most countable since idempotents are in one-to-one correspondence 
with subgroups of Z" generated by faces of S (the family of subgroups of Z" is 
countable since each of them can be generated by n elements of Z"). 

Example 5. The light cone — — q^ = in M.^ contains infinitely many 
straight lines passing through points in Z,^. Let the semigroup 5' C Z^ be defined 
by inequalities r > 0, — — q^ > 0. Then Ia is infinite. 

Perhaps, the most significant difference between finitely and infinitely generated 
algebras occurs in the following example. 

Example 6. Let a G R be irrational and put S — {{p, q) E Z'^ : p + qa > 0}. 
Then A can be realized as an algebra of almost periodic function which are 
bounded and analytic in the upper half-plane, with the set of almost periods in 
the group {p ~ qa : ip,q) G Z^}; A is antisymmetric, has no orthogonal real 
measures, and is a maximal subalgebra of C(T^). In [6, Ch. 7], these algebras 
are studied in details. 

Proposition 16. A compact Lie group G has the property (SW) if and only if 
its centre is finite. 

Proof. If Z{G) is not finite, then it contains a circle subgroup H = T; the family 
of all function in C{G) that admit analytic extension to D from each coset of 
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H is an invariant algebra on G which is not self-adjoint. The converse follows 
from Proposition [8] and Corollary [5] □ 



A connected compact Lie group G has a finite centre if and only if G is 
semisimple but in general its identity component may be even abelian. Propo- 
sition [16] can be easily generalized to all compact Hausdorff groups. A compact 
Hausdorff group is said to be profinite if any neighbourhood of its identity con- 
tains a normal subgroup of finite index. 

Theorem 16. A compact group G has the property (SW) if and only if its 
centre is profinite. 

Proof. Let Z be the centre of G, H he e. closed normal subgroup of G, and 
tt: G ^ G/H = P he the canonical homomorphism. If G satisfies (SW), 
then P also has this property (this is evident). By the structure theorems, 
any neighbourhood of the identity in G includes a subgroup H as above such 
that P is a Lie group. Since tt{Z) C Zp{P), ■k{Z) is finite by Proposition [TCI 
Hence, Z is profinite. Conversely, let G does not satisfy (SW). By Lemma [H 
then there exists a normal subgroup H such that P is a Lie group that does not 
satisfy (SW). Let Zp he the maximal torus of Zp{P); by Proposition [TBI Zp is 
nontrivial. We claim that Tr{Z) D Zp. Indeed, if G is a Lie group, then for each 
£. G 3p, where 3p is the Lie algebra of Zp, the affine space 7r~"^(^) is Ad(G)- 
invariant. Hence, it contains an Ad(G)-fixed point and the claim is true for Lie 
groups. The general case reduces to this one since the assumption tt{Z) 2 Zp 
remains true if one replaces G by its quotient group over some sufficiently small 
normal subgroup. □ 

Comparing Theorem [TBI and the result of J. Wolf who gave another char- 
acterization of these groups in [28], we get a corollary: a compact Hausdorff 
group has a profinite centre if and only if the image of each its one dimensional 
character is finite (of course, this can be proved directly, starting with the case 
of Lie groups). 

A function algebra A is called a Dirichlet algebra if A^ contains no nontrivial 
real orthogonal measure. In [52], D. Rider proved that a compact group which 
admits an antisymmetric Dirichlet algebra is connected and abelian. We prove 
a more precise version of this theorem for Lie groups. First, we describe a class 
of invariant algebras similar to the algebra of example [6] Let v G M" be such 
that 

{v, x)^0 for all x G Z" \ {0}. (67) 

This happens if and only if components of v are linear independent over Q. Set 

S^, ^{xe Z" : {v,x) > 0}. 

Let Ay he the invariant algebra on T" corresponding to the semigroup Sy . Then 
C = W^v and there are no other nontrivial cones in . If 7 is the complex 
ray corresponding to v, then 7(C^) is dense in A4a^, and 7(iR) is an irrational 
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winding of T". Hence, Ay may be identified with the algebra {/ o 7 : / 6 Ay} 
of analytic functions on C"*". Clearly, this algebra is generated by exponents 
ex{z) = e^^i""'^) ^ where x G Sy, z e C+. There are only two idempotents in 
M.A^'- e and e. Topologically, M.a^ is the product T" x [0,1] with identified 
points of the fibre T" x {0}. 

Theorem 17. Let G he a compact Lie group and A be an antisymmetric Dirich- 
let invariant algebra on G. Then G = T" for some n. The set Ta is finite and 
linearly ordered; each group Gj is a torus T"^ . Furthermore, if j < k are con- 
secutive idempotents iniA, then the algebra A\kGj is closed in G{kGj) and has 
the type Ay described above, where the vector v G M"^ satisfies (|67p . 

Proof. Let Gq be the identity component of G and Lq^^ be the averaging op- 
erator defined by p3p . If the algebra Lq^^A contains a nonconstant function, 
then it contains some real function since G/Gq is finite. Then A cannot be 
antisymmetric. If the averaging gives only constant functions, then the differ- 
ence between the Haar measures of G and Go is a real measure orthogonal to 
A. Hence, A is not a Dirichlet algebra if G ^ Gq. By Theorem [T4l and Theo- 
rem |8l Haar measures on G and its maximal torus are representing for e, hence, 
their difference is also a real orthogonal measure, which is nontrivial if G is not 
abelian. Therefore, G is connected and abelian. Since G is a Lie group, G = T" 
for some n. 

Set = ["^ -I- IntG*^. According to Theorem [T^ there exists a mapping 
77 : T Ma such that / o 77 is analytic and bounded on for all / G 
A. If dim ["^ > 1, then for any point in T'^ there exist different representing 
measures (for example, Poisson kernels on P and on some line in l^). For 
Dirichlet algebras, this is impossible; hence, dim G*^ = 1 and the closure of 
in A4a contains exactly one idempotent j ^ e. Therefore, each chain of rays 
which starts at e passes through j. Clearly, the averaging of a Dirichlet invariant 
algebra over each closed subgroup is again a Dirichlet one on the quotient group. 
Hence, the arguments above can be applied to the algebra RjA on the group 
G/Gj. Thus, Theorem 1131 implies that Xa is finite and linearly ordered. 

Since Gj is a p-set for A by Proposition [71 3), the restriction of A to Gj is 
closed in G{Gj). Obviously, A\q. is a Dirichlet invariant algebra. Hence, Gj is 
a torus. 

Thus, it remains to prove that A — Ay, where v £ M" satisfies (p7)) . assuming 
that Ia — {e,e}. Let v generate and 7 : C+ Ma be the corresponding 
complex ray. For any z G C^, there exists the unique representing measure for 
7(z) concentrated on 7(jR) (the projection of the Poisson kernel on R). The 
support of any weakly limit point of these measures is contained in the closure 
of 7(i]R). Since the Haar measure of G is the unique representing measure for 
e, 7(iR) is dense in G; i.e. it is an irrational winding of G. This property is 
equivalent to Set 5 = SpA C Z" = G. Then 5 C 5„ since S* ^ C" . If 

X, —X i S, then Re x -L ^; hence, 

5U(-S') ==Z". 

Therefore, S ^ Sy. □ 
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9 Invariant algebras on spherical homogeneous 
spaces 

Let Af = K\G be a right homogeneous space of a compact connected Lie group 
G, where K is the stable subgroup of the base point o. In this section, we 
consider G-invariant algebras on homogeneous spaces A/, assuming that M is 
multiplicity free: the quasiregular representation of G in C{M) contains every 
irreducible representation of G with a multiplicity < 1. This remarkable class 
of homogeneous spaces is characterized by each of the following properties (see, 
for example, 27J; if G is not compact, then they are not equivalent): 

1) M is commutative, i.e., the algebra of all invariant differential operators 
on M is commutative; 

2) {G,K) is a Gelfand pair, i.e., the convolution algebra of all left and right 
if -invariant functions in L^{G) is commutative; 

3) a generic G-orbit in the cotangent bundle T*M is coisotropic; 

4) M is weakly commutative (i.e., the Poisson algebra G{T*M) is commuta- 
tive) ; 

5) M is weakly symmetric; 

6) = G^/K^ is spherical, i.e., a Borel subgroup of G'^ has an open orbit 
in M^. 

We shall say that M is spherical. It is well known that the group G acting on 
itself by left and right translation is a spherical homogeneous space, GxG/G. In 
what follows, M is supposed to be spherical and A denotes a G-invariant closed 
subalgebra of G(M). Let Z = 2'i3ig(jvf)(G) be the group of all diffeomorphisms 
of M that commute with every g <E G. Clearly, any transformation in Z keeps 
each irreducible component of the quasiregular representation (which will be 
denoted by R). The Schur lemma and the definition of the multiplicity free 
spaces imply the following (well-known) assertion. 

Lemma 12. If M is spherical, then any G-invariant closed subspace of G{M) 
is Z -invariant. The group Z is abelian. □ 

Transformations in Z are induced by the left translation^ x — > gx, wher^ 
X € G and g belongs to the normalizer of K in G. Hence, Z and ZG — GZ 
are compact Lie groups. Thus, studying G-invariant algebras on M , we may 
assume without loss of generality that 

^mff(M)(G)CG, (68) 

where ^{jiff (m) (^) identity component of Z. Let Z = Zq{G) be the 

identity component of the centre of G and a be the Haar measure of Z. 
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Lemma 13. Let A be antisymmetric, H C G be a closed subgroup, and v be its 
Haar measure. If H D Z , then RyA ~ C, where C denotes the set of constant 
functions on M . 

Proof. It is sufficient to prove the lemma assuming H = Z: indeed, the evident 
equality R^Ra = Rv imply i?^ = R„ since both operators keep constant func- 
tions. By Theorem 8] (whose most essential part was proved in [T7]), if M 
admits no one parameter group of transformations commuting with the action 
of G, then each G-invariant algebra on M is self-adjoint with respect to the 
complex conjugation. Set K' = KZ; it follows from that the homogeneous 
space M' = K'\G has this property. The space R^A consists of all Z-invariant 
functions in A. Hence, it can be considered as a subalgebra of G(Af'). Since A 
is antisymmetric, we get RaA = C. □ 

Let _ff be a closed abelian subgroup and Spj:^ A be the set of weights of R 
for H in A. Since A is an algebra, Sp^ A is a semigroup in C{H). Hence, its 
closed linear span Bh is an invariant algebra on H . 

Lemma 14. Let H and Bh be as above, v be the Haar measure of H . Suppose 
that H D Z and A is antisymmetric. Then Bh is antisymmetric. 

Proof. It is sufficient to prove (UH). For x G Sp^ A, set 



If (|42|) is not true, then there exists a nontrivial character x S Sp^ A such that 
X e SpH A. Then 



Suppose H = Z. It follows from (|70p that there exist nontrivial G-irreducible 
subspaces U,V C A such that 



for all u £ U , V £ V , z E Z , X (E M. The product uv is Z-invariant. Hence 
Ra-{uv) = uv and uv = const by Lemma [T51 This implies dim(C/y) < 1. Since 
U and V are G-invariant and nontrivial, UV ^ 0. Thus, dim(L/y) = 1. Since all 
functions in U and V are real analytic, the assumption 7i,w 0, where u £ U 
and V £ V, implies uv ^ 0. Consequently, dim C7 = dimV^ = 1. It follows 
that the semisimple part of G acts on U and V trivially. Thus, ((7T|) . taken 
together with the structure theorems and the additional (nonessential) condition 
u(o) = v{o) = 1, imply v = u. Since A is antisymmetric, u ~ v ~ const. 
Therefore, x — const contradictory to the assumption. 

Let H D Z. According to the proven above, if x G Sp^ A and x G ^Ph 
then X = 1 on Z. Hence, any x-eigenfunction is Z-invariant. By Lemma [T3l it 
is constant. Thus, x is trivial. □ 




(69) 



P^,Py^O in A. 



(70) 



u{xz) — x{z)u{x), v{xz) = x{z)v{x) 



(71) 
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Corollary 10. Let A be antisymmetric, H and v be as in Lemma \1S\ Then 
the mapping R^, : A C is a homomorphism. 

Proof. By Lemma 1141 and Theorem [8l v is muhipHcative on the algebra Bh- 
Taken together with Lemma [T31 this proves the corollary. □ 

In fact, a stronger version of Corollary [TO] is true. Let H be as in Lemma 
and P be a closed subgroup of H such that 

P^nSpHA = {0}, (72) 

where P^ = {x & H : x|j3 = l}is the annihilator of P in H, and let >c be the 
Haar measure of P. 

Corollary 11. If (|72p holds, then R^ is a homomorphism of A onto C. 

Proof. Due to Corollary[TOl it is sufficient to prove that R^ = Ru in A, but this 
is a consequence of (|72|) . □ 

The following proposition generalizes to the case of spherical spaces an es- 
sential part of Theorem [51 We keep the notation above. Let a be the Haar 
measure of Z and ^ be the invariant measure on M such that /i(A/) = 1. 

Proposition 17. Following assertions are equivalent: 

1) A is antisymmetric; 

2) R„ is a homomorphism A — > C; 

3) /i is multiplicative on A; 

4) there is a G- fixed point in A4a. 

Proof. The implication 1) 2) is a particular case of Corollary [TO) If R^r is a 
homomorphism A C, then for all / G A and z E Z 

J fdfx = J RJdfi^ J jRJdfxda{z)= J R,fdti = R,f{o), 

where the first equality holds since /i is invariant and the last is true since 
= const. Hence, 2) implies 3). The corresponding to ^ point of Ma is 
G-fixed. Conversely, ii ly E A4a is G-fixed, then /i G J^i, since the set Aii, is 
weakly compact, convex, and G-invariant. Thus, 3) 4). It remains to note 
that M is a set of antisymmetry if 3) is true since M = supp /x and /i is a 
representing measure. □ 

We use the assumption that M is spherical only in the proof of the implica- 
tion 1) ^ 2). Without this assumption, the implication is false. For example, it 
is not true for adjoint orbits M — Ad{G)v and algebras A ~ P{M) (the closure 
of polynomials on M), where G = SU(2), v = ih. + re, r > 0, and h, e, f is the 
standard sl2-triple (cf. [H] or [TO]). 

The canonical projection tt : G M — K\G induces an embedding tt : 
C{M) — ^ C{G). In the following theorem, we identify A and i^{A), assume (|68)) . 
and keep the notation of this section. 
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Theorem 18. Let M be spherical, A he antisymmetric and B he the closed 
(hi)invariant algebra on G generated by A. Then B is antisymmetric and the 
action of G on M extends to the action of the semigroup A4b on A4a in such 
a way that for any (p G Mb the mapping defined by 

R^fW^fi-ipv) for alH, e Ma, f e A, (73) 

is an endomorphism of A. Moreover, = Rf^ for any /i € M^. If e is zero of 
Mb, then R^A = C. 

Proof. Let T be a maximal torus in G and Bt be as in Lemma [U (for H = T). 
Clearly, x G Spj^ A if and only if there exists / ^ in ^ such that f{mt) = 
f{m)x{t) for all m £ M, t £ T. It follows that Sp^ ^ is a semigroup. As a 
closed linear space, B is generated by products of left shifts of functions in A. 
Since the left shifts commute with the right ones, they do not change Sp^^ A; the 
products also have spectrum in Spj^ A because Spj^ ^ is a semigroup. Therefore, 

Spt^ B — Spjn A. 

Hence, Bt coincides with the closure in C(T) of the linear span of characters 
in Spj^ A. By Lemma [HI and Theorem [HI B is antisymmetric. It follows from 
Lemma [7] that any ip S IIom(Spgi A, D) defines an endomorphism R^p : A^ A 
by setting 

on Afin. Identifying Hom(Spyyl,B) with Mbt according to Theorem [31 we 
get an action of T on that satisfies ([75]). where the right side defines the 
left one. This action extends to Mb due to Theorem [TUI and Theorem [T51 a). 
Clearly, for any fixed ip G Mb and all f E A, the function Ljpf{g) = f{tpg), 
g E G, belongs to B. Integrating by g over fi e M^, we get R^ = R^^. The last 
assertion follows from Theorem [U c). □ 

Probably, the action of Theorem [T5I is transitive (i.e. Ma = pMb for 
any p G M). Due to Theorem SI this is true if A is the averaging of B over 
K. For spheres U(n)/U(ri — 1), the latter was proved in [9] by methods of 
harmonic analysis (which is rather simple on them) . The description of invariant 
algebras on these spheres can be found in [26], their maximal ideal spaces are 
characterized in [16]. 

We conclude with an infinite dimensional version of the Hilbert-Mumford 
criterion for commutative homogeneous spaces. It admits several equivalent 
forms for finite dimensions. We formulate a version which is a bit stronger than 
the classical one and extends to the infinite dimensional case. 

Let G C GL(y) be a compact connected Lie group, q be its Lie algebra, and 
let V € V. Suppose that 

(N) p(v) ~ for any G-invariant holomorphic homogeneous polynomial p of 
positive degree. 
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Then there exists € g such that 



lim e'*^v = 0. (74) 

t — >+oo 

In the standard statement for the field C, the assertion concerns an algebraic 
reductive group. Any such a group is the complexification of a compact group 
G; if T is a maximal torus in G, then has the Cartan decomposition G^ = 
GT^G. This makes it possible to reduce the problem to the compact case. We 
omit the proof; the exposition below is self-contained. 

Here is a simple example which shows that the criterion is not longer true 
for Banach spaces, in the above form, in any sense. We define a homogeneous 
polynomial p of degree n on a Banach space V by 

p{v) = (f>{v, ...,v), 

where (f> is a, continuous n-linear form on V. A polynomial is a finite linear 
combination of homogeneous ones (including constants), V denotes the set of 
all polynomials on V. 

Example 7. Let A be any antisymmetric (bi)invariant algebra on a compact 
Lie group G and set 

V = {feA: /(e) = 0}, 

where e is the zero of A4a- Thus, V is the unique G- invariant maximal ideal 
in A. Clearly, V is 9JtA-invariant. For every G-invariant function p on V, any 

V € V and each ray 7 G EH*^ the function q{it) — p{R-y{it)v) does not depend 
on t £ K since 7(it) lies in G. If p is holomorphic, then q{z) = p{v) for all 
z G Therefore, p{Rjv) — p{v) for j — 7(00). The arguments above hold for 
the vector RjV and each ray 7 G 9^-'; applying them repeatedly, we get 

p{v) =p{R^v) =_p(0) 

due to Theorem 1131 It follows that each G-invariant polynomial is constant; 
thus, (N) is true for all v . 

Let ^ e satisfies ([74l) for a generic v and T be a torus which contains 
exp(R^). Then ([74|) holds for P^^i there x is a character of T and is the 
projection to the ^-isotypical component. Clearly, Spy v ~ Sp^^ A for a generic 

V G A; then (|71)l implies that ^ G G*^. Thus, exp(ji^) defines a ray 7 G IH^ such 
that limt_^+oo = ( but there exist algebras which does not admit such a ray 
(see Example S]). □ 

Perhaps, this means that one parameter semigroup in a right statement 
of any infinite dimensional version of the Hilbert-Mumford criterion must be 
replaced by a chain of them. We prove this for the spherical orbits (condition 
([55]) is not assumed). Operators ^ G fl are well-defined on the space Vfin of 
vectors v such that the linear span of Gv is finite dimensional. 
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Theorem 19. Let G be a compact connected Lie group acting strongly continu- 
ously on a Banach space V . Suppose that v &V satisfies (N) and that the orbit 
M = Gv is a spherical homogeneous space. Then there exist a non-increasing 
sequence Vi, . . . ,Vn of closed subspaces of V , vectors Vk G Vfe, where 

fc = 1, . . . , 71 + 1, Vi ^ V, Vn+l = 0, 

and pairwise commuting ^i, . . . , G such that and e'*^'', t > 0, are well- 
defined on a dense subspace of Vk and 

1) for any t > 0, e**^'' extends to a continuous operator Vk Vk; 

2) limf^+ooe**^''Wfe =Wfc+i, k = l,...,n. 

Proof. Let A be the closure of P in C(Af ). Clearly, A is an invariant algebra 
on M. Averaging any p ^ V over G, we get a polynomial (this is an easy 
consequence of the definition of a homogeneous polynomial) which is necessarily 
G-invariant. It follows from (N) that 

p(0) — Jj^jpd^ for all p <eV, 

where ^ is the normalized invariant measure on M — Gv (note that the converse 
is also true). Therefore, n is multiplicative on V, hence on A. Since supp ji — M, 
A is antisymmetric (see the proof of Theorem [8|). 

Let K C G he the stable subgroup of v, N be its normalizer in G, and H be 
a maximal torus in N. The group of all transformations of AI which commute 
with G may be identified with N/K acting on M by right; by Lemma \T2\ N/K 
is abelian. Let Z be the identity component of N/K acting by right on M . Set 
G = ZG, H = ZH and let i>, k be the Haar measures of H, H, G, respectively. 
Clearly, H is a torus which contains the identity component of the centre of G; 
hence we may apply Lemma fT4l and Corollary [10] to G. Since Hv = LLv, we get 
for any p E V 

/ p{hv) dv{h) = I p{hv)di){h)— j p{gv) dk{g) = I pdii — p{0). 
Jh Jh Jg Jm 

Therefore, (N) holds for H and v. Thus, we may assume H — G, i.e., that G is 
a torus. Set Vq = V, vi = v, 

Si^SpHVi={xeH : P^vi^O}, 

where is defined by ([69]) . and let Vi be the closure of J^xeSi ■ Then 

Spff Vi ^ Sph vi C ^■ 

By Corollary [21 the representation of G in Vi extends to the representation of 
M.A which is strongly continuous. Let 71 be a ray in A4a such that 71 (0)111 = vi 
and V2 — 7i(oo)wi ^ vi. Due to Theorem [Til there exists ^1 £ f) such that 
li{t)vi ~ exp{itS^i)vi for all i > 0. Then (N) holds for V2 (see the begin- 
ning of Example [7]). Hence, the invariant measure on Hv2 is multiplicative on 
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V and on its closure A2- Applying this procedure repeatedly and replacing 
Vk, Vk by Wfc+i, Vfc+i, respectively, we get a sequence of vectors Vk such that 
dunHvk > dim. Hvk+i, if Vk+i 7^ Vk (see Lemma [5]). Let d„ be the first one 
that satisfies Vn+i = Vn+2- Since the closure An+i of the restriction ofV to the 
torus Hvn+i is an antisymmetric invariant algebra, Theorem [T3] and Theorem[5] 
imply Hvn+i ~ {I'n+i}- If i^n+i 7^ 0, then the averaging over H of any contin- 
uous linear functional that does not annihilate w„+i gives an i/-invariant linear 
functional with the same property, contradictory to (N) . □ 
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